mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
fix(managed-agent): stop a bound Turn under refused authorization (#13163)
* feat(managed-agent): admit later Turns of a Workspace-bound Session for its creator G0 admits only the initial file-tool Turn created with a Workspace-bound Session; every later public submit is refused with workspace_unavailable, so a Hosted Session cannot hold a conversation. Admit a later Turn when the deployment's Workspace-files opt-in is on and the submitter created the Session. Execution already authorizes each Turn against the creator's Workspace grants (WorkspaceExecutionStore.authorize), so letting another actor submit would run tools under the creator's authority; every other actor keeps the existing refusal (404 without read access, 409 otherwise). The store admits a bound Session's later Turn only under the same opt-in. Cancel, rename, lifecycle and cwd operations stay gated. - ManagedWorkspaceRegistry.createdSession looks up the creator in managed_workspace_create_command. - HostedPublicWorkspaceIT: a reader who is not the creator is refused; the creator's second Turn runs write, edit and read again through the real Broker and worker and completes. - ManagedWorkspaceAdmissionTest: the enabled store admits the later Turn and the creator lookup distinguishes actors and tenants. - The Workspace-binding capability description (contract and generated WebShell types), the README and both G0 design documents describe the new boundary. The contract version is left for #13101, which takes 1.25. Not built or run locally. * feat(web-shell): let the creator send later Turns to a bound Session The server now admits later Turns from a Workspace-bound Session's creator, but the Managed panel still hid the composer for every bound Session and the Java provider forced canSend to false. - WebShellSessionCapabilities gains workspaceTurns, computed per Session and caller with the same rule the submit path enforces (opt-in on, caller can read, caller created the Session). The contract and the generated WebShell types describe it. - The Java provider sends for a bound Session only when the service reports workspaceTurns, and reports it in the summary only when true. - The Managed panel shows the composer for such a Session; cancel stays hidden for bound Sessions. - HostedPublicWorkspaceIT checks the capability per caller through the WebShell route; the contract test's WebShell capability shape and provider/page tests cover the client. * feat(managed-agent): let the bound Session creator cancel a running Turn Later Turns of a Workspace-bound Session could be submitted by the creator but not cancelled: the service and store refused the cancel, and the coordinator skipped bound Sessions entirely. Apply the same rule to cancellation. The creator, with the Workspace- files opt-in on, may cancel; the coordinator then cancels through the Hosted Harness like any other Turn, which aborts the Turn and settles its Runtime calls through their original identities. Other actors and deployments without the opt-in keep the existing refusal, and the coordinator still leaves a bound Session alone without the opt-in. - HarnessCoordinatorTest: without the opt-in only the opt-in is read; with it, the bound Turn is cancelled through the Harness. - HostedPublicWorkspaceIT: a later Turn is held in the model call; a reader's cancel is refused, the creator's cancel is accepted, and the Turn ends CANCELLED with no tool execution. - WebShell reports canCancel for such a Session when workspaceTurns is set, so the Managed panel shows Cancel. - Contract text, generated types, README and both G0 design documents now keep only lifecycle operations gated. * feat(managed-agent): let the bound Session creator rename it Rename of a Workspace-bound Session answered 409 until a role source decided who may change it (D4 4.9). #12867 settled the first version: the Session's creator is its only owner. Rename changes only the title through the Hosted Harness and touches no Runtime, so apply the same creator rule under the Workspace-files opt-in; the store admits only a RENAME mutation of a bound Session, and only under the opt-in. Close, archive, delete and unarchive stay gated: the embedded Broker's drain only stops warming a closed Session and has no Harness-level teardown yet, so closing a bound Session would leave its worker and any held Workspace lease behind. HostedPublicWorkspaceIT: a reader's rename is refused; the creator's rename returns the new title. * style(web-shell): keep the workspaceTurns expression on one line as Prettier prints it * test(managed-agent): read the renamed title from the public Session metadata PublicSession carries its title under metadata.title; the rename assertion read a top-level title and saw an empty string although the rename returned 200. * fix(web-shell): match the generated optional workspaceTurns capability * test(managed-agent): keep the approval run out of the later-Turn checks * docs(managed-agent): Align G0 gating statements with creator later-Turn admission The G0 section and Prerequisites still published the pre-follow-up gate ("later submit/cancel remain gated"), contradicting the boundary paragraph this PR adds; state the creator's submit/cancel/rename admission and keep close/archive/delete/unarchive and cwd gated. Scope the follow-up's refusal precisely: requireLegacyWorkspace answers workspace_unavailable only when the actor can read the Workspace, and session_not_found otherwise. Apply the same two corrections to the bilingual design doc and note the follow-up's composer UI change. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuoob5v90j * chore(managed-agent): Bump the public API contract to 1.27.0 The deferral resolved: #13101 merged and took 1.25.0, and #13117 took 1.26.0. Record this PR's contract change per the version-history convention: creator later-Turn admission and the workspaceTurns capability. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuoob5v90j * fix(web-shell): hide execution-unavailable banner line when creator can send later Turns The workspace-binding banner always stated that message execution is unavailable, contradicting the composer this PR enables for the creator of a bound Session with workspaceTurns capability. Gate the third banner paragraph on !capabilities.workspaceTurns so the page no longer states both. Pin both branches in ManagedSessionsPage tests. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuoqgbnv0l * test(sdk-java): pin bound-Session admission clauses with negative controls Three of the four clauses this PR adds to bound-Session admission could be deleted with the suite staying green: - ManagedWorkspaceRegistry.createdSession's session_id column: create a second bound Session by actor-b under a distinct idempotency key and assert createdSession(tenant, actor-a, otherSession) is false, so the lookup must match this Session, not any Session the actor created. - maySubmitWorkspaceTurn's harness.isWorkspaceFilesAvailable() clause: assert the bound-session web-shell GET reports capabilities.workspaceTurns == false while the opt-in is off. - boundRenameAllowed's kind conjunct: assert beginSessionMutation with UNARCHIVE on the enabled store throws workspace_unavailable, and add the positive insertCancelCommand control so the cancel path stays open for bound Sessions under the opt-in. Each new assertion was verified to go red under the corresponding mutation (session_id dropped, flag clause dropped, kind conjunct dropped, cancel conjunct dropped) and the suite returns green with the sources restored. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuoqgbnv0l * test(sdk-java): exercise later Turns under approval-mode=default and pin canRead HostedPublicWorkspaceIT changes: - register() now grants the reader in both runs (can_create only in the files run), removing the access-table primary-key collision that forced the approvals run to skip the later-Turn block. - The approvals run now drives a later Turn through answerActions and asserts it completes, so a later Turn admitted under approval-mode=default is covered; the cancel/rename probes keep their held model reply and stay in the files run to fit the method timeout. - proof.txt is reset to a sentinel just before the later-Turn submit, so the post-Turn "after" assertion can fail if the later Turn's write and edit stop reaching the bound root (R1-13). - After the rename check, revoking the creator's read grant asserts submit, cancel and PATCH all answer 404 session_not_found, pinning the canRead clause of maySubmitWorkspaceTurn (R1-12 clause 3). - Model-request counts become 16 (approvals) / 18 (files) and the approvals run now answers 8 actions across the two Turns. Compile-verified via mvn test-compile. Execution requires the bundled dist/cli.js plus a MySQL/hosted-Harness stack, which this environment cannot provide; behavior was traced against requireSubmitter / requireLegacyWorkspace / boundRenameAllowed and the fixture request model, and CI remains the executor. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuoqgbnv0l * fix(sdk-java): align bound-Session later-Turn admission with the execution authority Round-2 review of #13112 found the admission predicate certifying Turns that execution can never run. maySubmitWorkspaceTurn now also requires the predicates the cited authority fixes at creation (Session ACTIVE, the qwen-code agent, the frozen execution profile refs) and the creator's can_create on an ACTIVE registry, read through the same grant row shape; a can_read-revoked creator still falls through to the documented 404. Cancellation attaches passively so an abort no longer depends on the physical mount still verifying. renameSession answers a non-retryable refusal with its own status and code instead of a transient 503. requireSubmitter loads the Session once per call, and the session read paths skip the canRead probe their entry already established. The OpenAPI contract, README and both design docs now state the creator admission, its grants qualifier and the still-gated lifecycle/cwd operations consistently, and the generated WebShell types are regenerated from the corrected contract. Tests pin the bound rename admission, the two empty-creation profile escapes and the can_create-revoked arm. Co-authored-by: Qwen-Coder <[email protected]> * fix(sdk-java): cancel a live bound Turn through its running attachment26de98cd7emade the live cancel attach passively. A passive attach reloads the Session in the Hosted Harness instead of reusing the running Turn's attachment, so the abort never reached the Turn and HostedPublicWorkspaceIT timed out with the Turn still CANCELLING. Restore the ordinary attach for the live path; cancellation recovery, which has no live attachment, keeps attaching passively. Cancelling under a refused Workspace authorization stays a follow-up. * fix(managed-agent): stop a bound Turn under refused authorization Follow-ups to #13112 from its round-4 verification (#13162). Cancelling aborts work that is already running, so it no longer needs the grants that admit new work: the creator who can still read the Workspace may cancel while can_create is revoked, the Workspace is draining or it was re-registered. A live cancel reuses the running Turn's attachment and runs no Workspace authorization; only without one does it fall back to the existing attach. A cancel the Harness did not take is re-sent with backoff while the Turn is still CANCELLING, since the running dispatcher checks CANCELLING only once. Admission of new work now also requires the Workspace generation and storage the Session was bound to, so a re-registration refuses submit and rename synchronously, before any command row is written. Tests pin the opt-in clause, the creator cancel under revocation and re-registration, the live-attachment path and the resend, and the IT cancels a running Turn after revoking the grant and draining the Workspace. * fix(managed-agent): fail the Turn when a cancel attach is refused The bound-Session cancel path attaches through the strict 3-arg createOrLoad. When the Workspace authority, or the storage guard behind it, refused that attach, the RuntimeException fell into the catch that promises recovery: no harness.cancel was issued for a cancel the API had already answered 202 for, no sweeper re-claims a Turn whose lease the live consumer keeps renewing, and the Turn ran on and settled COMPLETED. Settle the Turn with the refusal's own code instead of dropping the cancel silently. Addresses review thread R2-1 (PRRT_kwDOPB-92c6n-tVW). Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmupkgkl21b * fix(managed-agent): retire a refused rename command instead of wedging rename The permanent-refusal branch in renameSession rethrew a 4xx after beginSessionMutation had already written a PENDING RENAME_SESSION row. Nothing retires that row, so every later rename with a fresh key died in requireNoOpenOperation with session_operation_active for the Session's life. abandonSessionMutation deletes the still-PENDING row before the refusal is answered, so the Session accepts the next rename again. Addresses review thread R3-1 (PRRT_kwDOPB-92c6n-tVl). Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmupkgkl21b * refactor(managed-agent): drop the dead read-grant operand and fix the published rule maySubmitWorkspaceTurn's canRead operand, readGranted flag and 2-arg/3-arg overload pair cannot change any result: findReadable already joins the same access row with can_read = TRUE, and createdSession still runs first so an actor id the registry key cannot encode keeps answering false instead of throwing. The README's later-Turn rule now names the create grant and the registry's ACTIVE state, as the OpenAPI capability text already does. Addresses review threads R1-8 (PRRT_kwDOPB-92c6n-tVt) and R1-5 (PRRT_kwDOPB-92c6n-tV0, PRRT_kwDOPB-92c6n-tV-). Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmupkgkl21b * refactor(managed-agent): cancel an admitted Turn without attaching bindHarness compares the given boot with the Session's bound boot, and an admitted Turn's boot is already bound, so the attach in cancelAdmittedTurn only recomputed it while re-running the Workspace authorization. Pass the bound boot instead and drop liveAttachment; harness.cancel already reuses the running Turn's attachment. Re-send at a fixed two-second interval instead of a backoff table. * docs(managed-agent): state the full later-Turn admission rule and narrow the bound-Session notice The published rule on createSession, webShellCreateSession and the workspaceTurns description omitted the caller's live read and create grants and the registry's ACTIVE state, so the public surface could not discover the rule maySubmitWorkspaceTurn actually applies; the public descriptions now also say there is no per-caller flag there and callers must handle 409 workspace_unavailable. The bound-Session notice claimed message execution "is not available in this service yet", which is false for every per-caller refusal (not the creator, grant revoked, Workspace draining) while the deployment serves the creator fine; narrow the copy to "You cannot send messages in this Session" so it is true for both causes. The webShellCreationIsMetadataOnlyUntilExecutionIsWired comment claimed to pin the isWorkspaceFilesAvailable clause though its 3-arg registration's drifted profile refs make the refusal over-determined; say so instead. Addresses review threads R3-4 (PRRT_kwDOPB-92c6n-tWn), R3-6 (PRRT_kwDOPB-92c6n-tWw) and the R3-2 comment anchor (PRRT_kwDOPB-92c6n-tWZ). Co-authored-by: Qwen-Coder <[email protected]> * fix(managed-agent): deliver cancellation through the running owner * fix(ci): bound hosted browser dependency installation * fix(ci): record the ci.yml growth in the workflow size baseline The hosted-browser install step adds two lines to ci.yml, taking it to 147136 bytes: 4110 over the recorded 143026 and 14 past the 4096-byte allowance, which is why "Check workflow file size" failed. Most of that growth is main-side drift the allowance had been absorbing, but this PR touches the file, so the ratchet's stale-baseline leniency does not apply and the number has to move in the same PR. The growth is real — a bounded apt/Playwright install — so bump the entry instead of shrinking the workflow. Verified locally: .github/scripts/check-workflow-size.sh exits 0 against basea7deb01bcb, and scripts/tests/workflow-size.test.js passes 220/220. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmupvw3d00b * fix(ci): reserve browser smoke time after slow dependency downloads * fix(ci): keep time for smoke after slow hosted installs * test(web-shell): wait for step expansion before selecting next group * test(ci): align hosted browser timeout contract * fix(managed-agent): retry transient cancel-attach refusals instead of settling the Turn The cancelAdmittedTurn catch settled the Turn permanently for every RuntimeException out of createOrLoad, so a transient blip (a retryable broker refusal, a daemon 5xx, a store error) terminally failed an already-admitted, still-executing Turn whose Harness side kept running. Rethrow anything that is not a non-retryable RuntimeBrokerException so the Turn stays CANCELLING and the dispatch sweep re-attempts the cancel, the pre-existing recovery path. Only a permanent refusal still settles the Turn, under its own broker code. That classification makes cancelRefusalCode's non-broker fallback unreachable — a RuntimeBrokerException code is non-null by constructor invariant — so the helper is removed rather than tested as dead code. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuq5w6bb0f * fix(managed-agent): retire the rename command row on every answered failure A rename answered without completing retired its PENDING command row only for non-retryable broker refusals; every other failure family (a daemon 4xx/5xx, the connector's approval-mode IllegalStateException) left the row PENDING forever, and requireNoOpenOperation then wedged every later lifecycle change of the Session with 409 session_operation_active. Retire the row for any RuntimeException and keep the retryable/permanent split only for choosing the answered status. The freed key also stayed poisoned: the requested event the abandoned attempt published has a source_key derived from the key, so a same-key re-attempt collided on UNIQUE (tenant_id, session_id, source_key) and answered 500. beginSessionMutation now skips re-appending an already published requested event, matching the no-event replay it replaces. Two existing tests pinned the wedge (a 409 for a different key after an answered failure, and a replay header on the same-key retry); they now assert admission and a fresh, non-replay re-attempt. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuq5w6bb0f * docs(managed-agent): qualify Workspace cancel contract * docs(web-shell): scope Session creation capabilities * docs(managed-agent): state the retired-rename-command contract in the README Two README passages still described behavior this PR changed. The lifecycle paragraph promised a failed rename leaves a resumable PENDING command, but abandonSessionMutation now deletes that row, so a same-key retry is a fresh attempt answered replayed=false and the requested event the abandoned attempt published is not re-appended. The W1a paragraph still denied the public next-turn admission that the G0 section and the v1.28 OpenAPI note now describe; narrow it to the creator's later Turns under the opt-in and keep the resume clause. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuqn1gku0t * test(managed-agent): pin the command half of requireNoOpenOperation allowsOneLifecycleChangeAtATime held the suite's only witness for the PENDING-command conjunct, and this PR replaced those four lines because they pinned the wedge the retirement removes. Deleting the conjunct from requireNoOpenOperation now survives the whole module suite, so build a PENDING command row through the store and assert a close and a delete both answer 409 session_operation_active while no operation row is open. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuqn1gku0t * test(web-shell): pin the bound-Session Cancel control at the page level This PR moved the Cancel button inside the composer <form> and made that form's visibility depend on `!summary?.workspace || workspaceTurns`. No page-level fixture was both bound and cancellable (0 of 23), so neither half of that composition was observable here: reverting the form gate, or re-adding `!summary.workspace` to the Cancel condition, left every case in the file green while a bound Session's creator lost the ability to stop a Turn they had just started. Adds one case mirroring the unbound cancel test. Measured both mutants: reverting the gate to `!summary?.workspace` and adding `!summary.workspace` to the Cancel condition each turn this case red (1 failed / 33 skipped) while the unmutated tree passes 34/34. Refs review finding R1-10. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuqp6mdb01 * fix(managed-agent): answer same-key retries from the record, retire rows on any permanent rename failure - submitTurn/renameSession: look up the recorded admission before the Workspace-admission gate, so a same-key retry of an admitted submit or a completed rename is answered with the recorded outcome even after a re-registration / revoked grant / DRAINING flip the gate off (R1-7). A PENDING rename row still falls through to beginSessionMutation, which answers it as replayed and re-drives the unfinished mutation. - renameSession: classify the retire-on-refusal branch by permanence, not by exception type — the connector's approval-mode IllegalStateException and permanent (<500) DaemonHttpException wedge the PENDING row exactly like a non-retryable broker refusal (R1-16). - OpenAPI: workspaceTurns is served on every response, so move it into WebShellSessionCapabilities.required (R1-5). Tests are written but not executed locally (no JDK 21/Maven on this host); the three new ManagedWorkspaceAdmissionTest cases go red without their fix by construction. Co-authored-by: Qwen-Coder <[email protected]> * fix(managed-agent): clear rename admission when harness is disabled * fix(managed-agent): repair the two red required lanes on this head Both failures are this PR's own, not the base's: - ManagedAgentService.renameSession called CommandRecord.commandStatus(), which does not exist; the record's component is status. This is the sole cause of both Java lane failures (maven-compiler-plugin:compile, "cannot find symbol / method commandStatus()"), and lines 304 and 363 of the same file already use the correct "COMPLETED".equals(command.status()) idiom. - WebShellSessionCapabilities.workspaceTurns moved into the OpenAPI required array, but the generated TypeScript mirror was not regenerated, so the contract-drift guard in managed-agent-api.test.ts failed (expected `workspaceTurns: boolean;`, checked-in `workspaceTurns?: boolean;`). Regenerated with `node scripts/generate-managed-agent-api.mjs`; the only line that changed is the one the guard reported. Verified locally: managed-agent-api.test.ts 2/2 pass, java-managed-agent-provider.test.ts + ManagedSessionsPage.test.tsx 53/53 pass, eslint clean on the regenerated file. The Java edit could not be compiled here (no JDK 21 / Maven on this host); it is a single symbol-name correction read off the record declaration in this tree. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuqtgxwd07 * test(managed-agent): expect the closed Session status for a queued bound Turn WorkspaceSessionCloseMySqlIT (#13135) was written before this PR, when a bound Session refused every later Turn with workspace_unavailable. With Workspace files enabled this PR admits the creator's later Turns, so the Turn queued behind the close now reaches the Session status and answers session_not_active, which is what the test means to show: the close commits before the queued admission inspects the Session. * test(managed-agent): read the renamed Session title from metadata The public Session body projects the Session title into metadata.title (ManagedAgentService.publicSession); PublicSession has no title component, so the three rename assertions written against title() broke testCompile and reddened both Java lanes on this head. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmur1oq2c0i * fix(managed-agent): restore admission error precedence in CI * fix(managed-agent): keep the Workspace refusal ahead of the Harness gate Both red required Java lanes fail the same six assertions on MySQL 8.4 and MariaDB (run 37022561191, jobs 110889014145 / 110889014257). Both causes are in this branch's production code; main has not touched packages/sdk-java since the merge baseb3dda468f2. - submitTurn ran requireHarness() ahead of requireSubmitter(), so a Session the Workspace refuses was answered with 503 hosted_harness_disabled instead of 409 workspace_unavailable whenever the Harness is not configured. That is the four ManagedWorkspaceAdmissionTest failures (revocationHidesBoundSessionAndBlocksRetry:372, emptyBoundCreationOutsideTheProfileIsNotAdmittedForLaterTurns:663, creatorCancelsWithoutTheGrantsThatAdmitNewWork:1028, reRegistrationRefusesLaterWorkBeforeAnyCommandIsWritten:1055). Restore the order cancelTurn and renameSession already use, keeping the same-key replay lookup ahead of both gates so R1-7 still holds. - renameSession classified every IllegalStateException as a permanent failure, retiring the PENDING command row and answering 409 session_mutation_refused. Only the attach path is permanent: that is where QwenHostedHarnessConnector refuses an unconfirmed Session approval mode (lines 133 and 364). An IllegalStateException out of harness.rename() is an unreachable Harness, and main pins it as a transient 503 hosted_harness_unavailable whose row a same-key retry re-drives (ManagedSessionLifecycleTest .allowsOneLifecycleChangeAtATime:382, ManagedAgentServerIntegrationTest .retriesAPendingRenameWithTheSameIdempotencyKey:660, both through the failNextRename fixture at ManagedAgentServerIntegrationTest:1744). Track whether the attach returned and narrow the arm to that call site, which keeps permanentNonBrokerRenameFailureRetiresItsCommandRow green without widening the transient path. Not compiled or executed here: this host has no mvn and only JDK 1.8 (`which mvn` empty, `java -version` = 1.8.0_322, no ~/.m2), while the lanes run JDK 21 with surefire/failsafe. The change is confined to two statements and their guard; CI is the verifier. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmur4wgp80n * fix(runtime-broker): close drained lost workspace bindings * docs(managed-agent): state the cancel rule and refusal codes the code enforces The public createSession description bound cancel to the submit-grade conditions that requireCanceller deliberately does not check (create grant, ACTIVE registry row, active undeleted frozen-profile Session), so an integrator gating its Cancel control on them would withhold a cancel the server accepts and could not abort a runaway Turn on a draining or re-registered Workspace. Split the sentence: submit and rename keep the submit-grade rule, now also naming the generation/storage conjunct, and cancel states the narrower creator-plus-Workspace-read rule. Name the 404 session_not_found that requireLegacyWorkspace answers when the caller lost Workspace read, next to the 409 already documented. Drop the rename promise from webShellCreateSession - the WebShell surface exposes no rename route, rename is updateSession on Public Sessions only - re-landing the removal already made at7720f6dfda. Add the ACTIVE registry-state conjunct to the workspaceTurns capability so it matches maySubmitWorkspaceTurn and the "same rule" claim beside it. The generated WebShell mirror is updated to match; the generator imports openapi-typescript, which is unavailable without node_modules here. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmurtwo5e1r * fix(managed-agent): answer bound replays only to the creator and keep Cancel reachable R3-1: submit and rename now check the creator's durable creation receipt before the replay lookup, so a reader who did not create a bound Session neither replays the creator's admission nor learns which keys exist. The receipt survives revocation, draining and re-registration, so the creator's own same-key retry still replays. R3-2: cancel is no longer tied to workspaceTurns in the WebShell provider, and the Cancel button renders outside the composer when the composer is hidden. Send keeps its gate; the server refuses a cancel from anyone else. R3-3, R3-4: scope the published cancel claim to a resident attachment (after a restart or takeover the owner re-attaches through Workspace execution authority and the Turn may stay cancelling while it refuses), and align the README's G0 twin with the later-Turn rule. * fix(managed-agent): keep a retired rename from reverting a newer title completeSessionMutation accepts a FAILED receipt so a same-key sibling that entered the Harness before the retirement can still complete. Nothing ordered that completion: while the receipt was FAILED, requireNoOpenOperation no longer saw it, so a rename under a fresh key could begin and complete, and the late sibling then overwrote that newer title with a 200 and a second session.updated event. Refuse to complete a FAILED receipt when a mutation of the same kind completed after its requested event, answering 409 session_mutation_superseded. Mutation commands on a Session begin one at a time, so event sequence ids order this strictly where millisecond timestamps could tie. A same-key request sent after the newer rename revives the receipt to PENDING and still applies, so the newest request wins as before. Reported on #13112 (ManagedAgentStore.java:565 review thread). * style(web-shell): format the bound Cancel page test with Prettier * style(web-shell): apply prettier to ManagedSessionsPage.test.tsx The Lint & Static job failed on `node scripts/lint.js --prettier` with "Code style issues found in 1 file" for packages/web-shell/client/components/managed/ManagedSessionsPage.test.tsx. Reformat that one expectation so the branch matches the repo prettier style. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmus7hcqa2c * fix(managed-agent): recover cold-cache cancels without new-work grants * docs(managed-agent): include storage in both capability descriptions * refactor(managed-agent): delete the review-minted dead surface and its duplicate Closes four open review findings by deletion only; the diff shrinks by 92 lines and no production behavior changes. - R2-3: both convenience constructors on WebShellSessionCapabilities had zero call sites. The only construction is the canonical 8-arg call in ManagedAgentService.webShellSession; the contract test touches the record through .class for schema naming. The 2-arg overload also took (tasks, actions) while the record's components run (tasks, artifacts, actions), so it silently mapped its second argument to `actions`. - R2-4: isSessionCreator was a one-line wrapper over createdSession with the two String parameters reversed, leaving two public registry methods that differ only in argument order on a creator-identity authorization check. Deleted the wrapper; its single caller in ManagedAgentStore.requireWorkspaceCreator now calls createdSession with the declared (tenantId, actorId, sessionId) order. Equivalent by construction: the wrapper body was that same call. - R2-5 and R2-6: aRenameFailureThatIsNotABrokerRefusalStillRetiresItsCommand and permanentNonBrokerRenameFailureRetiresItsCommandRow both pin the same single branch -- renameSession's catch fall-through to abandonSessionMutation plus dependencyUnavailable("hosted_harness_unavailable") -- with the same three assertions. They were distinct while the catch still classified DaemonHttpException(<500) and IllegalStateException separately; retiring the row on every answered failure collapsed them. The deleted twin also stubbed its DaemonHttpException double with a bare mock(), so getStatusCode() returned 0 rather than the documented 4xx, and its TransactionTemplate wrapper mirrors no production contract (renameSession carries no @Transactional). The surviving test uses the shared boundSession/boundServiceWith helpers, which build the identical workspaceFilesEnabled configuration and the identical bound Session. The two imports only that test used are removed, as checkstyle enforces UnusedImports for this module. Not compiled or executed here: this host has no mvn and only JDK 1.8. Every removed member was grepped to zero remaining references across the module, and no surviving import was left unused. The Java lanes on the pre-push head52704a9de0are green and are the verifier. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmusjmlf72u * fix(managed-agent): dedupe by dropping this PR's own added test, not main's Corrects8d98903588, which deleted the wrong twin. aRenameFailureThatIsNotABrokerRefusalStillRetiresItsCommand is pre-existing main coverage: it sits at line 776 of ManagedWorkspaceAdmissionTest.java both at this PR's merge base2a6879e648and on origin/main, introduced by #13112 (2b15eac862), which is an ancestor of the merge base. Removing it made this PR delete 80 lines of unrelated main test coverage. permanentNonBrokerRenameFailureRetiresItsCommandRow is the twin this PR added, and it is the one that duplicates main's test: both drive the same single branch (renameSession's catch fall-through to abandonSessionMutation plus dependencyUnavailable("hosted_harness_unavailable")) with the same three assertions. They were only distinct while the catch classified DaemonHttpException(<500) and IllegalStateException separately; retiring the row on every answered failure collapsed them. So the deletion moves to the PR's own addition. The test file's diff against the merge base is now 233 insertions and zero deletions instead of 291/80. The two imports8d98903588removed (DaemonHttpException, Mockito.mock) are restored, as main's test still uses both; no import is left unused, which this module's checkstyle enforces. The R2-3 (unused WebShellSessionCapabilities constructors) and R2-4 (isSessionCreator wrapper with reversed String parameters) deletions from8d98903588are unchanged and unaffected. Not compiled or executed here: this host has no mvn and only JDK 1.8. The removed method was grepped to zero remaining references and the deletion is brace-balanced; the Java lanes are the verifier. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmusjmlf72u * fix(serve): retain passive recovery leases on refusal * fix(managed-agent): keep the re-registration guard on the page twin The page refactor on main rewired webShellSession(session, actorId) to the batch assembler, whose maySubmitWorkspaceTurn twin answers the creator submit gate from the batched grant read alone. That read carried no binding stamp, so a re-registered Workspace kept advertising workspace_turns on the page and on the single-session path while the singular gate still refused the submit through bindingCurrent. Carry workspace_generation and storage_id in the batch grant read and compare them with the Session's recorded binding, so the twin answers the same rule without a per-row registry query: the grant batch stays the one managed_workspace_registry read the page budget asserts. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmutypl6357 * fix(managed-agent): keep the Session lifecycle term on cancel admission requireCanceller hand-copied only the workspace and files disjuncts of maySubmitShape, dropping the Session lifecycle term that main enforced through requireSubmitter. A cancel on a CLOSED, ARCHIVED or DELETED bound Session answered 202 and wrote a CANCEL command where main answered 409 workspace_unavailable and wrote nothing. Cite maySubmitShape instead: it restores the lifecycle term and removes the duplication. The grant and registry terms stay out of the cancel rule, so the creator can still cancel while can_create is revoked, the Workspace is draining or it was re-registered. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuu9s91d5p * fix(managed-agent): record a passive adoption stranded by a concurrent teardown The resident passive-load path answered its two post-await exits without noteOwedAdoption, even though onPassiveRuntimeAcquired had already recorded the adopted Runtime lease. close() admits a concurrent DELETE here: its guard is only active/mcpBusy/mcpRecovering/hooksBusy, none of which a parked passive recovery sets, and the route allows a missing client id. Its releaseLeaseNow then runs while runtimeLeaseHeld is still undefined, so it releases nothing, the acquire resolves afterwards, and the adopted Runtime Session is left unreachable from every route with nothing named on stderr. Record the strand on both exits rather than releasing it: a release persists the record as RELEASED and every retried acquire of the same identity then conflicts with 409 runtime_session_not_acquirable. Also drain refusedAdoptions on both resident drive-redrive success exits, matching the resident passive success exit and the record's own contract that the next successful load of the id drains it. Without the drain a later genuinely stranded adoption is suppressed by the has(sessionId) latch. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuug7qaz61 * fix(managed-agent): fence every resident reattach and keep its refusal retryable The resident split answered a redriven load from the same resident connection as a passive one, but only the passive branch revalidated the store identity. Hoist that fence above both branches so a foreign tenant or Workspace stays hidden and a drifted Session Store address is refused on either path. A 404 is settled as a terminal Turn failure by the only caller, which would drop an admitted cancellation on the storage-drift refusal; answer the retryable conflict instead. The drive branch also resumed from its recovery await with no post-await revalidation, so a teardown admitted during the await answered 200 with the detached Session's clientId and left the acquired lease unreachable. Mirror the passive branch's fences there and record the owed adoption. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuva7z827l * fix(managed-agent): fence the parked passive recovery against teardown The resident passive load publishes its adopted Runtime lease onto the route-reachable Session the moment broker.acquire() resolves, while the recovery is still reading execution states. close() fenced only active/mcpBusy/mcpRecovering/hooksBusy, none of which a parked passive recovery set, and DELETE /session/:id resolves with a missing client id allowed, so a teardown landing in that window released a lease that was still mid-adoption. The release persists the record RELEASED, and RuntimeBrokerService.acquireNewSession answers 409 runtime_session_not_acquirable for any stored state that is neither READY nor ACQUIRING, so every later cold takeover and every cancel-path status/cancel for that parked Turn failed and the Workspace stayed pinned. Hold close()'s own fence for the whole recovery rather than recording the strand afterwards: the teardown is refused with 409 hosted_turn_active, nothing is released mid-adoption, and the adopted lease goes back with the teardown the fence lets through. The pre-diff teardown released nothing but left the adoption unreachable from every route; the fence removes that window too, so the two post-await exits no longer carry a false comment. Also states the deployment's Workspace-files opt-in in the README's authoritative cancel rule: requireCanceller reaches maySubmitShape, which returns false the moment harness.isWorkspaceFilesAvailable() is false, before falling through to requireLegacyWorkspace, which answers workspace_unavailable for a bound Session. The other contract artifacts already state the condition. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuvp83ny8e * fix(managed-agent): preserve concurrent recovery and rename ordering Count all pending passive recovery requests before permitting teardown. Persist the newest mutation attempt boundary so a failing sibling does not reject its successful retry. Co-authored-by: Qwen-Coder <[email protected]> * fix(test): import Mockito times in coordinator tests Co-authored-by: Qwen-Coder <[email protected]> * fix(serve): preserve live cancellation reattachment Skip ownerless cancellation settlement while the original Turn still runs, so a cold Java attachment can reattach passively and cancel that owner. Co-authored-by: Qwen-Coder <[email protected]> * fix(serve): restore resident recovery outcomes Preserve terminal decline reasons and reattach requested approvals. Settle payable terminal projections without retrying settled file history, and refuse a cancellation attachment removed during settlement. Co-authored-by: Qwen-Coder <[email protected]> * fix(managed-agent): retry transient approval mount failures Use delivery-time mount verification for action responses and recover cold approval attachments passively after rechecking new-work authority. Preserve shared acquire verdicts and permanent workspace refusals. Co-authored-by: Qwen-Coder <[email protected]> * fix(managed-agent): renumber the mutation-attempt migration to V51 main published V48__workspace_storage_migration.sql plus V49/V50 while this branch was open, and the merge that pulled them in left two files claiming version 48, so scripts/check-flyway-migrations.js fails the "Flyway migration version uniqueness" gate. V51 is the next free version across both migration locations (SQL and BaseJavaMigration, whose highest is V29). Nothing refers to the file by name or version: the code and tests only use the mutation_attempt_sequence column, and Flyway applies every migration in the location, so the renumber only moves this ALTER after main's V48-V50. Verified locally with the same command the gate runs: node scripts/check-flyway-migrations.js packages/sdk-java/managed-agent-server \ packages/sdk-java/runtime-broker packages/sdk-java/qwencode before: "2 migrations claim version 48" (rc=1); after: "51 migrations, all versions unique" (rc=0). Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmuy7yk8zcm * test(managed-agent): expect the V51 mutation-attempt migration in the upgrade IT WorkspaceMigrationMySqlIT.upgradesCurrentMainWithoutChangingAppliedMigrations pins every migration applied after the V47 baseline, so renumbering V48__managed_mutation_attempt_sequence.sql to V51 left that expectation one version short and the MariaDB failsafe job failed 1 of 58 tests. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmuya3q1ncp * test(managed-agent): keep Hosted fixture ports distinct Co-authored-by: Qwen-Coder <[email protected]> * fix(managed-agent): retry operator-reversible refusals on the action-response path A revoked creation grant or a registry that left ACTIVE is operator-mutable state, not the structural verdict the terminal exit promises: stamping the committed approval decision FAILED over it certifies a confirmation the Harness never saw, and restoring the grant cannot recover it. The passive attachment authority now splits its verdict into structural, grant-only, and authorized; only the action-response delivery maps the grant-only refusal to a retryable workspace_unavailable so the delivery machine re-arms while recovery waits for the operator. Every other caller — creation, passive load, cancellation — keeps the previous terminal refusal. * fix(cli): refuse a resident inapplicable redrive while a prompt slot is active The helper re-validated registration and closing after its awaits but never that the Session was still idle, and the latch clear removed the only incidentally closing guard: a files/rewind admitted inside the window had its prompt slot overwritten by the settle projection and its AbortController dropped while /status reported the Session idle. Add the third post-await re-validation the continue and managed-runtime cancel routes already perform, answering the session-scoped refusal before the projection can run. * test(managed-agent): pin V52 after renumbering the mutation-attempt migration * fix(managed-agent): preserve approval retries during cold attachment * test(managed-agent): pin the legacy receipt rewrite and the acquire verdict control - Wrap the two raw jdbc receipt mutations in exactly-one-row assertions so a 0-row arm can no longer collapse onto the tested path. - Add the positive acquire-path control after the action-response probe retry proves its retryable refusal: the same mount must still authorize without a verdict change. Review threads on the 2026-10-07 automated closeout round. * test(sdk-java): allow heartbeat recovery time after refused detach Keep the confirmed-detach observation at 200ms, but allow rejected detach responses two seconds to demonstrate continued HTTP heartbeats on macOS. Co-authored-by: Qwen-Coder <[email protected]> * test(hosted): pin teardown and pending rename recovery guards Add HTTP-level deletion and closing interleavings for resident inapplicable recovery, and a public-store pending rename witness. Keep the inherited main heartbeat deadline fix unchanged. Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: yiliang114 <[email protected]> Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: yiliang114 <[email protected]>
This commit is contained in:
32 files changed
+2666
-396
No files matched your search
@@ -26,6 +26,31 @@ Turn against the creator's Workspace grants, so any other actor, and every
|
||||
deployment without the opt-in, keeps the existing refusal:
|
||||
`workspace_unavailable` when the actor can read the Workspace,
|
||||
`session_not_found` when they cannot. The creator may also rename the Session.
|
||||
Admitting new work requires the creator's create grant on an `ACTIVE`
|
||||
Workspace at the generation and storage the Session was bound to, so a
|
||||
re-registration refuses submit and rename before any command is written.
|
||||
Cancelling only aborts work already running: while the deployment still enables
|
||||
Workspace files, the creator who can still read the Workspace may cancel even
|
||||
after the create grant is revoked, the Workspace starts draining or it is
|
||||
re-registered. A live cancel reuses the running Turn's resident attachment.
|
||||
A cold connector cache re-attaches for the persisted cancellation, validating
|
||||
its frozen Session binding and exact tenant/Session identity without requiring
|
||||
mutable creation grants, registry state or mount readiness. New API requests
|
||||
still require the creator's read grant; a cancellation already accepted keeps
|
||||
retrying if that grant is later revoked. New work always rechecks execution
|
||||
authority, including when physical recovery is disabled. Passive load of a
|
||||
resident Harness Session returns the original client identity after validating
|
||||
its tenant, Workspace, Session Store URL and frozen profile. It does not reopen
|
||||
the writer or drive work; an inactive parked Runtime Turn is reported again if
|
||||
a prior load reply was lost. Passive recovery may adopt the original Runtime
|
||||
and query status, but does not prepare or execute work. On the cancellation
|
||||
path, the adopted lease stays owed through lost replies and retryable refusals
|
||||
until terminal success or teardown. This does not prove recovery after Broker/worker
|
||||
process death or resolve an original prompt admission whose reply was lost.
|
||||
A cancel the Harness did not take is re-sent while the Turn is still cancelling. After each
|
||||
successful lease renewal, the running owner observes cancellation requested
|
||||
through any API replica and sends it on the executor, keeping network waits
|
||||
off the lease scheduler. Failed deliveries retry at the lease renewal interval.
|
||||
Workspace close follows its separate close capability and lifecycle admission.
|
||||
Archive, delete and unarchive follow the separate retention capabilities after
|
||||
reliable Workspace close. Cwd operations remain gated for bound Sessions.
|
||||
@@ -46,7 +71,7 @@ reliable Workspace close. Cwd operations remain gated for bound Sessions.
|
||||
checks, Session creation, initial Turn and actor-scoped idempotency remain in
|
||||
the existing creation transaction. Replays preserve the original identities
|
||||
and binding; changed payloads conflict.
|
||||
- Before attaching a bound Session, the connector rechecks the persisted binding
|
||||
- Before attaching a bound Session for new work, the connector rechecks the persisted binding
|
||||
through `WorkspaceExecutionStore.authorize`. Broker acquisition and execution
|
||||
retain their own grant, generation, storage and ownership checks. No failed
|
||||
binding falls back to the global Workspace or an unbound no-tool Session.
|
||||
@@ -56,21 +81,22 @@ reliable Workspace close. Cwd operations remain gated for bound Sessions.
|
||||
- Cold load of unsettled input remains blocked. G0 does not enable in-flight
|
||||
continuation, adopt workers, remove affinity or change the G1 failover gates.
|
||||
|
||||
- A live cancellation reuses its admitted Harness attachment and is retried by the current lease owner. It never certifies a terminal failure from a fresh attach refusal. Recorded rename failures retain a `FAILED` command receipt and digest; same-content retries are replays, conflicting content remains rejected, and a concurrent success can complete the retained receipt.
|
||||
- A cancellation reuses its admitted Harness attachment or passively re-attaches from a cold connector cache and is retried by the current lease owner. It never certifies a terminal failure from a fresh attach refusal. Recorded rename failures retain a `FAILED` command receipt and digest; same-content retries are replays, conflicting content remains rejected, and a concurrent success can complete the retained receipt.
|
||||
|
||||
## Changes and ownership
|
||||
|
||||
| Layer | Change | Scope |
|
||||
| ----------------------------------- | ------------------------------------------------------------------------ | ---------------------------------------- |
|
||||
| Java configuration | Explicit file admission opt-in and dependency validation | Deployment |
|
||||
| Creation service and SQL store | Admit initial input only under fixed, authorized Workspace configuration | Tenant, creator and persisted Workspace |
|
||||
| Coordinator | Dispatch admitted bound Turns only when the opt-in is enabled | Persisted Session and leased Turn |
|
||||
| Java connector and private SDK DTOs | Resolve the Session binding and pass the profile on create/load | Persisted Session and live Harness owner |
|
||||
| Existing Broker/worker | Reuse production routing and fencing | Selected Runtime and persisted Workspace |
|
||||
| Contract and README | Document the narrow creation capability and remaining gates | Public REST and WebShell adapter |
|
||||
| Layer | Change | Scope |
|
||||
| ----------------------------------- | ------------------------------------------------------------------------------------ | ---------------------------------------- |
|
||||
| Java configuration | Explicit file admission opt-in and dependency validation | Deployment |
|
||||
| Creation service and SQL store | Admit initial input only under fixed, authorized Workspace configuration | Tenant, creator and persisted Workspace |
|
||||
| Coordinator | Dispatch admitted bound Turns only when the opt-in is enabled | Persisted Session and leased Turn |
|
||||
| Java connector and private SDK DTOs | Resolve the Session binding and pass the profile on create/load | Persisted Session and live Harness owner |
|
||||
| Hosted private load route | Reuse resident connection after frozen identity checks; report parked Turn passively | Live Session owner |
|
||||
| Existing Broker/worker | Reuse production routing and fencing | Selected Runtime and persisted Workspace |
|
||||
| Contract and README | Document the narrow creation capability and remaining gates | Public REST and WebShell adapter |
|
||||
|
||||
Production behavior changes under `packages/sdk-java/managed-agent-server`, in
|
||||
the private Hosted DTOs in `packages/sdk-java/qwencode`, and in the WebShell
|
||||
the private Hosted DTOs in `packages/sdk-java/qwencode`, in the CLI Hosted Session routes (`packages/cli`), and in the WebShell
|
||||
managed Sessions page and its providers (`packages/web-shell`); it covers the
|
||||
initial Workspace Read/Write/Edit Turn and the creator's later-Turn submit,
|
||||
cancel and rename admission. No core authority, tool
|
||||
@@ -111,7 +137,11 @@ while cwd remains gated), and unbound
|
||||
no-tool regression paths.
|
||||
|
||||
Focused SDK serialization, connector, store/admission and coordinator tests
|
||||
cover create/load identity, authorization rechecks and disabled gates. Run the
|
||||
cover create/load identity, authorization rechecks and disabled gates. The real
|
||||
Hosted stack must cancel after creation authority is revoked and the connector
|
||||
cache is cleared; generation-only and storage-only drift must refuse new work
|
||||
before any command is written. New cancellation requests after read revocation
|
||||
must remain hidden, while previously accepted cancellations still retry. Run the
|
||||
Hosted integration on H2 locally and include it in the existing Hosted MySQL CI
|
||||
suite. Record separately whether local MySQL is available. Build, typecheck,
|
||||
bundle, focused tests and two clean diff audits precede completion.
|
||||
@@ -125,3 +155,7 @@ creator, above), lifecycle enablement,
|
||||
distributed provisioning and W0e/G1–G3 recovery remain separate. The existing
|
||||
`EmbeddedRuntimeBroker` is a production component and remains allowed; the E2E
|
||||
must not replace it or bypass admission with direct store calls.
|
||||
|
||||
The late-rename supersession check protects the public SQL title and receipt.
|
||||
It runs after the Harness title write, so it does not order overlapping Harness
|
||||
writes. That inherited lifecycle issue remains tracked in #13269.
|
||||
@@ -10,18 +10,18 @@ Hosted Read/Write/Edit 已能经过生产 Broker 和 worker 执行,但目前
|
||||
|
||||
G0 开放随创建会话准入的一次初始文件工具轮次,使用现有公开 REST 路由及共享 service 的 WebShell 创建适配器。发现接口仍仅广播 Workspace 绑定能力,不宣称完整的 Workspace 执行支持。G0 本身无需修改 UI;后续改动唯一的 UI 变化是为创建者开放输入框与取消控件。
|
||||
|
||||
后续改动在同一开关下为会话创建者开放后续 Turn,并允许创建者取消正在运行的 Turn,由 Hosted Harness 中止该 Turn 并按原始 Runtime 身份结算。执行时每个 Turn 都按创建者的 Workspace 授权校验,因此其他 actor 以及未开启该开关的部署仍得到现有拒绝:actor 可读取该 Workspace 时为 `workspace_unavailable`,不能读取时为 `session_not_found`。创建者也可以重命名该会话。工作区关闭通过独立的关闭能力及生命周期准入控制;归档、删除与取消归档遵循可靠工作区关闭后的独立保留能力;cwd 操作仍保持门禁。
|
||||
后续改动在同一开关下为会话创建者开放后续 Turn,并允许创建者取消正在运行的 Turn,由 Hosted Harness 中止该 Turn 并按原始 Runtime 身份结算。执行时每个 Turn 都按创建者的 Workspace 授权校验,因此其他 actor 以及未开启该开关的部署仍得到现有拒绝:actor 可读取该 Workspace 时为 `workspace_unavailable`,不能读取时为 `session_not_found`。创建者也可以重命名该会话。准入新工作要求创建者在 `ACTIVE` 的 Workspace 上持有创建授权,且 Workspace 的 generation 与存储与会话绑定时一致,因此重新注册后提交和改名会在写入任何命令之前被拒绝。取消只中止已在运行的工作:部署仍开启 Workspace 文件能力时,仍能读取该 Workspace 的创建者,即使创建授权被撤销、Workspace 进入 drain 或被重新注册,也可以取消。正常取消复用运行中 Turn 驻留在内存中的挂接。连接器缓存丢失后,按已持久化的取消请求重新挂接,验证冻结的 Session 绑定和精确的 tenant/Session 身份,不再要求可变的创建授权、registry 状态或挂载就绪。新的 API 请求仍要求创建者的读取授权;已受理的取消即使随后失去读取授权也继续重试。新工作始终重新验证执行授权,包括关闭物理恢复功能时。对 Harness 中驻留会话的 passive load,在验证租户、Workspace、Session Store URL 和冻结 profile 后返回原始 client 身份,不重新打开 writer 或驱动工作;若之前的 load 回复丢失,会再次报告未运行但仍停驻的 Runtime Turn。被动恢复可以采用原 Runtime 并查询状态,但不会 prepare 或 execute;取消路径中,已接管的租约跨丢失回复和可重试拒绝保持待归还,直到终态成功或拆除会话。这不证明 Broker/worker 进程死亡后的恢复,也不解决原始 prompt 准入回复丢失的问题。Harness 未接收的取消会在 Turn 仍处于取消中时重发。每次成功续租后,执行 owner 检查任意 API 副本受理的取消请求,并在 executor 中发送,避免网络等待阻塞续租调度器。投递失败按续租间隔重试。工作区关闭通过独立的关闭能力及生命周期准入控制;归档、删除与取消归档遵循可靠工作区关闭后的独立保留能力;cwd 操作仍保持门禁。
|
||||
|
||||
## 决策
|
||||
|
||||
- 部署显式启用 `harness.workspace-files-enabled`(环境变量 `QWEN_MANAGED_AGENT_WORKSPACE_FILES_ENABLED`),默认关闭。它要求 Hosted Harness、HTTP Session Store,以及同机、会话隔离的 local-process Broker。原有无绑定的无工具会话行为不变。关闭开关后拒绝携带输入的创建请求(包括重试);空输入的绑定会话创建和读取保持可用。
|
||||
- 仅接受 `qwen-code` 及现已支持并冻结的 `managed-runtime-tools/1` / `preapproved-workspace-tools/1` Workspace 配置组合。服务端选择 `hosted-workspace-files/1`。公开请求不能选择 profile,也不能通过 metadata 提升权限。
|
||||
- Workspace 解析、ACTIVE 状态、创建者读取/创建权限、固定 profile 校验、会话创建、初始 Turn 和 actor 范围的幂等继续位于现有创建事务内。重试保留原身份与绑定;载荷改变产生冲突。
|
||||
- 连接有绑定的会话之前,connector 通过 `WorkspaceExecutionStore.authorize` 重新检查持久绑定。Broker 获取与执行仍保留各自的权限、代数、存储和所有权检查。任何绑定失败都不能回退到全局 Workspace 或无绑定的无工具会话。
|
||||
- 为新工作连接有绑定的会话之前,connector 通过 `WorkspaceExecutionStore.authorize` 重新检查持久绑定。Broker 获取与执行仍保留各自的权限、代数、存储和所有权检查。任何绑定失败都不能回退到全局 Workspace 或无绑定的无工具会话。
|
||||
- 私有 create 与 load 都传递所选 profile 及持久 Workspace ID,包括创建冲突和创建结果不明后回退到 load 的路径。Harness 现有的不可变 definition 检查固定 profile。
|
||||
- 冷加载未结算输入仍被阻塞。G0 不启用在飞续接、接管 worker、取消 owner 粘性,也不改动 G1 failover 门禁。
|
||||
|
||||
- 取消运行中的 Turn 复用已准入的 Harness 连接,并由当前租约 owner 重试,不根据重新连接的拒绝伪造终态失败。已记录的重命名失败保留 `FAILED` 命令回执与摘要;相同内容重试仍是重放,不同内容继续冲突,并发成功请求仍可完成该回执。
|
||||
- 取消运行中的 Turn 复用已准入的 Harness 连接,或在连接器冷缓存下被动重新挂接,并由当前租约 owner 重试,不根据重新连接的拒绝伪造终态失败。已记录的重命名失败保留 `FAILED` 命令回执与摘要;相同内容重试仍是重放,不同内容继续冲突,并发成功请求仍可完成该回执。
|
||||
|
||||
## 改动与归属
|
||||
|
||||
@@ -31,10 +31,11 @@ G0 开放随创建会话准入的一次初始文件工具轮次,使用现有
|
||||
| 创建 service 与 SQL store | 仅在固定且获授权的 Workspace 配置下接受初始输入 | 租户、创建者及持久 Workspace |
|
||||
| Coordinator | 仅在开关启用时派发已准入的绑定轮次 | 持久会话及持有租约的 Turn |
|
||||
| Java connector 与私有 SDK DTO | 解析会话绑定,create/load 传递 profile | 持久会话及存活 Harness owner |
|
||||
| Hosted 私有 load 路由 | 校验冻结身份后复用驻留连接,被动报告停驻 Turn | 存活 Session owner |
|
||||
| 现有 Broker/worker | 复用生产路由与 fencing | 所选 Runtime 及持久 Workspace |
|
||||
| 契约与 README | 记录有限的创建能力及剩余门禁 | 公开 REST 与 WebShell 适配器 |
|
||||
|
||||
生产行为在 `packages/sdk-java/managed-agent-server`、`packages/sdk-java/qwencode` 的私有 Hosted DTO,以及 WebShell 托管会话页及其 provider(`packages/web-shell`)中变化,覆盖初始 Workspace Read/Write/Edit Turn 与创建者后续 Turn 的提交、取消和重命名准入。不需要为 core authority、工具执行循环、数据库 schema 或公开请求字段新增抽象。
|
||||
生产行为在 `packages/sdk-java/managed-agent-server`、`packages/sdk-java/qwencode` 的私有 Hosted DTO、CLI Hosted 会话路由(`packages/cli`),以及 WebShell 托管会话页及其 provider(`packages/web-shell`)中变化,覆盖初始 Workspace Read/Write/Edit Turn 与创建者后续 Turn 的提交、取消和重命名准入。不需要为 core authority、工具执行循环、数据库 schema 或公开请求字段新增抽象。
|
||||
|
||||
合入的改动还涉及该范围之外的两处,均不增加运行时行为:
|
||||
|
||||
@@ -47,8 +48,10 @@ G0 开放随创建会话准入的一次初始文件工具轮次,使用现有
|
||||
|
||||
初始轮次必须在所选 Workspace 的相对 cwd 下写、编辑并读取文件,产生持久工具历史和恰好一个公开终态事件,且不改动 Harness 的诱饵目录。重复创建幂等键,验证相同 Session/Turn 且无额外模型/工具副作用。验证改变载荷冲突、未授权租户/actor 无法创建或读取、不支持的 profile 与不可用 Workspace 被拒绝,以及关闭开关后保持原门禁。覆盖共享 WebShell 创建适配器、实际发生变化的后续操作门禁(创建者的后续 Turn 提交、取消与重命名被放行;关闭与保留操作遵循独立能力,cwd 操作仍受限)和无绑定无工具回归路径。
|
||||
|
||||
SDK 序列化、connector、store/准入及 coordinator 的定向测试覆盖 create/load 身份、权限复核与关闭的门禁。本地通过 H2 跑 Hosted 集成,并加入现有 Hosted MySQL CI 套件;单独记录本地 MySQL 是否可用。完成前执行 build、typecheck、bundle、定向测试和两轮无发现的完整 diff 自查。
|
||||
SDK 序列化、connector、store/准入及 coordinator 的定向测试覆盖 create/load 身份、权限复核与关闭的门禁。真实 Hosted 链路必须在创建授权被撤销且连接器缓存清空后完成取消;只变 generation 或只变 storage 时,必须在写入任何命令之前拒绝新工作。撤销读取授权后的新取消请求仍必须隐藏会话,而已经受理的取消继续重试。本地通过 H2 跑 Hosted 集成,并加入现有 Hosted MySQL CI 套件;单独记录本地 MySQL 是否可用。完成前执行 build、typecheck、bundle、定向测试和两轮无发现的完整 diff 自查。
|
||||
|
||||
## 边界与待定事项
|
||||
|
||||
本次实现把 G0 放在 #12952 下;以后调整到 D 或 W 跟踪不改变契约,也不决定 G3 的范围。Shell、审批、D8 AgentDefinition、公开 profile 选择、后续 Turn(此后已对创建者开放,见上文)、生命周期开放、分布式供给及 W0e/G1–G3 恢复均另行推进。现有 `EmbeddedRuntimeBroker` 是生产组件,可以继续使用;E2E 不得替换它或通过直接调用 store 绕过准入。
|
||||
|
||||
晚到改名的 supersession 检查保护公开 SQL 标题与回执。该检查发生在 Harness 写入标题之后,因此不保证重叠 Harness 写入的顺序。这个继承的生命周期问题继续由 #13269 跟踪。
|
||||
@@ -7987,7 +7987,7 @@ describe('Hosted Harness no-tool session', () => {
|
||||
await headers(supertest(server).delete(`/session/${SESSION_ID}`));
|
||||
});
|
||||
|
||||
it('reports an aborted turn as cancelled to the Java event projector', async () => {
|
||||
it('reports an aborted live turn as cancelled after a Java-style cold reattach', async () => {
|
||||
const log = vi
|
||||
.spyOn(stdio, 'writeStderrLineSafe')
|
||||
.mockImplementation(() => {});
|
||||
@@ -8015,9 +8015,19 @@ describe('Hosted Harness no-tool session', () => {
|
||||
.send({ prompt, promptId: PROMPT_ID, payloadDigest });
|
||||
expect(admitted.status).toBe(202);
|
||||
await vi.waitFor(() => expect(state.model).toHaveBeenCalledTimes(1));
|
||||
const reattached = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: store(),
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
driveRuntimeRecovery: false,
|
||||
cancellationTakeover: true,
|
||||
});
|
||||
expect(reattached.status).toBe(200);
|
||||
expect(reattached.body.clientId).toBe(created.body.clientId);
|
||||
const cancelled = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/cancel`),
|
||||
).set('X-Qwen-Client-Id', created.body.clientId as string);
|
||||
).set('X-Qwen-Client-Id', reattached.body.clientId as string);
|
||||
expect(cancelled.status).toBe(204);
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
@@ -8837,10 +8847,52 @@ describe('Hosted Harness tool approvals', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('cancels a waiting approval through the cancel route and releases the Workspace', async () => {
|
||||
it('passively reattaches a live Turn and cancels it without opening another writer', async () => {
|
||||
const { server, clientId, answer, status } = await waitingSession();
|
||||
// The frozen binding fences both resident reattach branches: a foreign
|
||||
// tenant or Workspace meets the budget-bounded already_attached refusal,
|
||||
// while a drifted Session Store address refuses with the retryable
|
||||
// conflict its only caller needs to retry instead of failing the Turn.
|
||||
for (const [changed, expected] of [
|
||||
[{ tenantId: 'other' }, 409],
|
||||
[{ workspaceId: 'other' }, 409],
|
||||
[{ baseUrl: 'http://other-store.test' }, 409],
|
||||
] as const) {
|
||||
for (const recovery of [
|
||||
{ passiveManagedRuntimeRecovery: true },
|
||||
{ driveRuntimeRecovery: true },
|
||||
]) {
|
||||
const refused = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: { ...store(), ...changed },
|
||||
...recovery,
|
||||
});
|
||||
expect(refused.status).toBe(expected);
|
||||
}
|
||||
}
|
||||
await headers(supertest(server).post(`/session/${SESSION_ID}/load`))
|
||||
.send({ managedSessionStore: store(), toolProfile: files })
|
||||
.expect(409);
|
||||
await headers(supertest(server).post(`/session/${SESSION_ID}/load`))
|
||||
.send({
|
||||
managedSessionStore: store(),
|
||||
toolProfile: 'hosted-workspace-shell/1',
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
})
|
||||
.expect(409);
|
||||
const loaded = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: store(),
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
});
|
||||
expect(loaded.status).toBe(200);
|
||||
expect(loaded.body).toMatchObject({ clientId, approvalMode: 'default' });
|
||||
expect(loaded.body._meta).toBeUndefined();
|
||||
expect(await status()).toMatchObject({ hasActivePrompt: true });
|
||||
await headers(supertest(server).post(`/session/${SESSION_ID}/cancel`))
|
||||
.set('X-Qwen-Client-Id', clientId)
|
||||
.set('X-Qwen-Client-Id', loaded.body.clientId as string)
|
||||
.expect(204);
|
||||
await waitFor(async () =>
|
||||
expect(await status()).toMatchObject({
|
||||
@@ -9585,7 +9637,7 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
||||
* execute hangs until the owner is "crashed" via cancel, leaving the turn
|
||||
* unsettled in the journal.
|
||||
*/
|
||||
async function parkToolTurn(toolProfile = FILE_PROFILE) {
|
||||
async function parkToolTurn(retainOwner = false, toolProfile = FILE_PROFILE) {
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'warm').mockResolvedValue();
|
||||
acquireSpy = vi
|
||||
.spyOn(HostedWorkspaceBroker.prototype, 'acquire')
|
||||
@@ -9658,13 +9710,16 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
||||
},
|
||||
{ timeout: 10_000 },
|
||||
);
|
||||
const closed = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(closed.status).toBe(204);
|
||||
if (!retainOwner) {
|
||||
const closed = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(closed.status).toBe(204);
|
||||
}
|
||||
// Only the parked owner's own acquires are behind us; a takeover's
|
||||
// acquire must be visible to the asserting test.
|
||||
acquireSpy.mockClear();
|
||||
return { server, clientId: created.body.clientId as string };
|
||||
}
|
||||
|
||||
async function loadReplacement(passive = false, toolProfile = FILE_PROFILE) {
|
||||
@@ -9896,7 +9951,7 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
||||
it.each([FILE_PROFILE, 'hosted-workspace-files/2'])(
|
||||
'settles the parked execution on load and continues the turn (%s)',
|
||||
async (toolProfile) => {
|
||||
await parkToolTurn(toolProfile);
|
||||
await parkToolTurn(false, toolProfile);
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'execute').mockResolvedValue({
|
||||
executionStatus: 'success',
|
||||
responseParts: [{ text: 'written' }],
|
||||
@@ -10022,6 +10077,386 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
||||
},
|
||||
);
|
||||
|
||||
function failFinalAuthorization(failure: 'blocked' | 'exception') {
|
||||
const original =
|
||||
LocalManagedSessionAuthority.prototype.harnessRunAuthorization;
|
||||
const acquisitionsBefore = acquireSpy.mock.calls.length;
|
||||
let failed = false;
|
||||
return vi
|
||||
.spyOn(LocalManagedSessionAuthority.prototype, 'harnessRunAuthorization')
|
||||
.mockImplementation(async function (this: LocalManagedSessionAuthority) {
|
||||
if (acquireSpy.mock.calls.length > acquisitionsBefore && !failed) {
|
||||
failed = true;
|
||||
if (failure === 'exception') throw new Error('store hiccup');
|
||||
return { status: 'blocked', reason: 'missing_state' } as never;
|
||||
}
|
||||
return original.call(this);
|
||||
});
|
||||
}
|
||||
|
||||
it.each(
|
||||
(['blocked', 'exception'] as const).flatMap((failure) =>
|
||||
[false, true].map((retry) => ({ failure, retry })),
|
||||
),
|
||||
)(
|
||||
'releases a resident passive adoption after $failure, retry=$retry',
|
||||
async ({ failure, retry }) => {
|
||||
const { server, clientId } = await parkToolTurn(true);
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'status').mockResolvedValue({
|
||||
state: 'prepared',
|
||||
});
|
||||
const release = vi.mocked(HostedWorkspaceBroker.prototype.release);
|
||||
release.mockClear();
|
||||
const stderr = vi
|
||||
.spyOn(stdio, 'writeStderrLineSafe')
|
||||
.mockImplementation(() => undefined);
|
||||
const owedLines = () =>
|
||||
stderr.mock.calls.filter(
|
||||
([line]) => line.includes('stays owed') && line.includes(PROMPT_ID),
|
||||
);
|
||||
const passiveLoad = () =>
|
||||
headers(supertest(server).post(`/session/${SESSION_ID}/load`)).send({
|
||||
managedSessionStore: store(),
|
||||
toolProfile: FILE_PROFILE,
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
});
|
||||
const authorization = failFinalAuthorization(failure);
|
||||
const refused = await passiveLoad();
|
||||
expect(refused.status).toBe(409);
|
||||
expect(refused.body.code).toBe('hosted_turn_recovery_required');
|
||||
expect(acquireSpy).toHaveBeenCalledOnce();
|
||||
expect(authorization).toHaveBeenCalledTimes(
|
||||
failure === 'blocked' ? 4 : 2,
|
||||
);
|
||||
if (failure === 'blocked')
|
||||
expect(await authorization.mock.results[1].value).toMatchObject({
|
||||
status: 'blocked',
|
||||
reason: 'missing_state',
|
||||
});
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
const firstRefusalDiagnostics = owedLines().length;
|
||||
let repeatedRefusalDiagnostics: number | undefined;
|
||||
authorization.mockRestore();
|
||||
if (retry) {
|
||||
const loaded = await passiveLoad();
|
||||
expect(loaded.status).toBe(200);
|
||||
expect(loaded.body.clientId).toBe(clientId);
|
||||
expect(acquireSpy).toHaveBeenCalledTimes(2);
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
// Successful attachment drains the refusal's diagnostic record.
|
||||
const failedAgain = failFinalAuthorization(failure);
|
||||
expect((await passiveLoad()).status).toBe(409);
|
||||
repeatedRefusalDiagnostics = owedLines().length;
|
||||
failedAgain.mockRestore();
|
||||
}
|
||||
const closed = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(closed.status).toBe(204);
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
expect(
|
||||
(release.mock.contexts[0] as HostedWorkspaceBroker).runtimeSessionId,
|
||||
).toBe(PROMPT_ID);
|
||||
expect(firstRefusalDiagnostics).toBe(1);
|
||||
if (retry) expect(repeatedRefusalDiagnostics).toBe(2);
|
||||
},
|
||||
);
|
||||
|
||||
it('refuses a teardown that lands while a resident passive load is adopting', async () => {
|
||||
const { server } = await parkToolTurn(true);
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'status').mockResolvedValue({
|
||||
state: 'prepared',
|
||||
});
|
||||
const release = vi.mocked(HostedWorkspaceBroker.prototype.release);
|
||||
release.mockClear();
|
||||
const stderr = vi
|
||||
.spyOn(stdio, 'writeStderrLineSafe')
|
||||
.mockImplementation(() => undefined);
|
||||
const owedLines = () =>
|
||||
stderr.mock.calls.filter(
|
||||
([line]) => line.includes('stays owed') && line.includes(PROMPT_ID),
|
||||
);
|
||||
let finishAcquire!: () => void;
|
||||
const acquireGate = new Promise<void>((resolve) => {
|
||||
finishAcquire = resolve;
|
||||
});
|
||||
acquireSpy.mockImplementationOnce(() => acquireGate as never);
|
||||
let loading: Promise<supertest.Response> | undefined;
|
||||
try {
|
||||
loading = headers(supertest(server).post(`/session/${SESSION_ID}/load`))
|
||||
.send({
|
||||
managedSessionStore: store(),
|
||||
toolProfile: FILE_PROFILE,
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
await vi.waitFor(() => expect(acquireSpy).toHaveBeenCalledOnce(), {
|
||||
timeout: 10_000,
|
||||
});
|
||||
// The parked recovery holds close()'s fence for its whole await, so a
|
||||
// DELETE without a client id is refused instead of racing the adoption:
|
||||
// releasing mid-adoption persists the record RELEASED and every later
|
||||
// acquire of that identity answers 409 runtime_session_not_acquirable.
|
||||
const closed = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(closed.status).toBe(409);
|
||||
expect(closed.body.code).toBe('hosted_turn_active');
|
||||
finishAcquire();
|
||||
const loaded = await loading;
|
||||
expect(loaded.status).toBe(200);
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
expect(owedLines()).toHaveLength(0);
|
||||
} finally {
|
||||
finishAcquire();
|
||||
await loading;
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
it.each(['resolved', 'rejected'])(
|
||||
'keeps teardown fenced until overlapping passive loads finish (%s)',
|
||||
async (secondOutcome) => {
|
||||
const { server, clientId } = await parkToolTurn(true);
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'status').mockResolvedValue({
|
||||
state: 'prepared',
|
||||
});
|
||||
const release = vi.mocked(HostedWorkspaceBroker.prototype.release);
|
||||
release.mockClear();
|
||||
let finishFirst!: () => void;
|
||||
let finishSecond!: () => void;
|
||||
let rejectSecond!: (cause: Error) => void;
|
||||
const firstGate = new Promise<void>((resolve) => {
|
||||
finishFirst = resolve;
|
||||
});
|
||||
const secondGate = new Promise<void>((resolve, reject) => {
|
||||
finishSecond = resolve;
|
||||
rejectSecond = reject;
|
||||
});
|
||||
acquireSpy
|
||||
.mockImplementationOnce(() => firstGate as never)
|
||||
.mockImplementationOnce(() => secondGate as never);
|
||||
const passiveLoad = () =>
|
||||
headers(supertest(server).post(`/session/${SESSION_ID}/load`))
|
||||
.send({
|
||||
managedSessionStore: store(),
|
||||
toolProfile: FILE_PROFILE,
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
let firstLoad: Promise<supertest.Response> | undefined;
|
||||
let secondLoad: Promise<supertest.Response> | undefined;
|
||||
try {
|
||||
firstLoad = passiveLoad();
|
||||
await vi.waitFor(() => expect(acquireSpy).toHaveBeenCalledOnce(), {
|
||||
timeout: 10_000,
|
||||
});
|
||||
secondLoad = passiveLoad();
|
||||
await vi.waitFor(() => expect(acquireSpy).toHaveBeenCalledTimes(2), {
|
||||
timeout: 10_000,
|
||||
});
|
||||
const bothInFlight = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(bothInFlight.status).toBe(409);
|
||||
expect(bothInFlight.body.code).toBe('hosted_turn_active');
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
finishFirst();
|
||||
const first = await firstLoad;
|
||||
expect(first.status).toBe(200);
|
||||
expect(first.body.clientId).toBe(clientId);
|
||||
const prematureClose = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(prematureClose.status).toBe(409);
|
||||
expect(prematureClose.body.code).toBe('hosted_turn_active');
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
if (secondOutcome === 'rejected') {
|
||||
rejectSecond(new Error('store hiccup'));
|
||||
} else {
|
||||
finishSecond();
|
||||
}
|
||||
const second = await secondLoad;
|
||||
expect(second.status).toBe(secondOutcome === 'resolved' ? 200 : 409);
|
||||
const closed = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(closed.status).toBe(204);
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
expect(
|
||||
(release.mock.contexts[0] as HostedWorkspaceBroker).runtimeSessionId,
|
||||
).toBe(PROMPT_ID);
|
||||
const repeatedClose = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(repeatedClose.status).toBe(404);
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
} finally {
|
||||
finishFirst();
|
||||
finishSecond();
|
||||
await Promise.all([firstLoad, secondLoad]);
|
||||
}
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
it('refuses a teardown that lands after a resident passive load published its adoption', async () => {
|
||||
const { server } = await parkToolTurn(true);
|
||||
const release = vi.mocked(HostedWorkspaceBroker.prototype.release);
|
||||
release.mockClear();
|
||||
const stderr = vi
|
||||
.spyOn(stdio, 'writeStderrLineSafe')
|
||||
.mockImplementation(() => undefined);
|
||||
const owedLines = () =>
|
||||
stderr.mock.calls.filter(
|
||||
([line]) => line.includes('stays owed') && line.includes(PROMPT_ID),
|
||||
);
|
||||
// onPassiveRuntimeAcquired publishes the adopted lease as soon as acquire
|
||||
// resolves, and the recovery reads execution states after that: gating the
|
||||
// read holds the teardown in the published-but-unfinished window.
|
||||
const status = vi
|
||||
.spyOn(HostedWorkspaceBroker.prototype, 'status')
|
||||
.mockResolvedValue({ state: 'prepared' });
|
||||
let finishStatus!: () => void;
|
||||
const statusGate = new Promise<void>((resolve) => {
|
||||
finishStatus = resolve;
|
||||
});
|
||||
status.mockImplementationOnce(async () => {
|
||||
await statusGate;
|
||||
return { state: 'prepared' };
|
||||
});
|
||||
let loading: Promise<supertest.Response> | undefined;
|
||||
try {
|
||||
loading = headers(supertest(server).post(`/session/${SESSION_ID}/load`))
|
||||
.send({
|
||||
managedSessionStore: store(),
|
||||
toolProfile: FILE_PROFILE,
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
await vi.waitFor(() => expect(status).toHaveBeenCalledOnce(), {
|
||||
timeout: 10_000,
|
||||
});
|
||||
const closed = await headers(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
expect(closed.status).toBe(409);
|
||||
expect(closed.body.code).toBe('hosted_turn_active');
|
||||
finishStatus();
|
||||
const loaded = await loading;
|
||||
expect(loaded.status).toBe(200);
|
||||
// The adopted lease stays on the live Session: releasing it here would
|
||||
// answer 409 runtime_session_not_acquirable to every later acquire of
|
||||
// the same identity.
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
expect(owedLines()).toHaveLength(0);
|
||||
} finally {
|
||||
finishStatus();
|
||||
await loading;
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
it('records the owed adoption when a teardown strands a drive redrive', async () => {
|
||||
await parkToolTurn();
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'status').mockResolvedValue({
|
||||
state: 'prepared',
|
||||
});
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'execute').mockResolvedValue({
|
||||
executionStatus: 'success',
|
||||
responseParts: [{ text: 'written' }],
|
||||
} as never);
|
||||
const release = vi.mocked(HostedWorkspaceBroker.prototype.release);
|
||||
const stderr = vi
|
||||
.spyOn(stdio, 'writeStderrLineSafe')
|
||||
.mockImplementation(() => undefined);
|
||||
const owedLines = () =>
|
||||
stderr.mock.calls.filter(
|
||||
([line]) => line.includes('stays owed') && line.includes(PROMPT_ID),
|
||||
);
|
||||
const { server, loaded } = await loadReplacement();
|
||||
expect(loaded.status).toBe(200);
|
||||
release.mockClear();
|
||||
let finishAcquire!: () => void;
|
||||
const acquireGate = new Promise<void>((resolve) => {
|
||||
finishAcquire = resolve;
|
||||
});
|
||||
acquireSpy.mockImplementationOnce(() => acquireGate as never);
|
||||
let redriving: Promise<supertest.Response> | undefined;
|
||||
try {
|
||||
redriving = replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
)
|
||||
.send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
driveRuntimeRecovery: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
await vi.waitFor(() => expect(acquireSpy).toHaveBeenCalledOnce(), {
|
||||
timeout: 10_000,
|
||||
});
|
||||
// The close route fences an active Turn, MCP work and Hooks — not a
|
||||
// parked redrive whose await has not recorded its lease yet.
|
||||
await replacementHeaders(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
).expect(204);
|
||||
finishAcquire();
|
||||
const redriven = await redriving;
|
||||
expect(redriven.status).toBe(404);
|
||||
expect(redriven.body.code).toBe('hosted_session_not_found');
|
||||
// The teardown handed the lease it could see; the redrive's own
|
||||
// adoption is named instead of released, because a release would
|
||||
// persist RELEASED and wedge every retried acquire of the identity.
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
expect(owedLines()).toHaveLength(1);
|
||||
} finally {
|
||||
finishAcquire();
|
||||
await redriving;
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
it.each(['blocked', 'exception'] as const)(
|
||||
'records a cold passive adoption after final authorization is %s',
|
||||
async (failure) => {
|
||||
await parkToolTurn();
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'status').mockResolvedValue({
|
||||
state: 'prepared',
|
||||
});
|
||||
const release = vi.mocked(HostedWorkspaceBroker.prototype.release);
|
||||
release.mockClear();
|
||||
const stderr = vi
|
||||
.spyOn(stdio, 'writeStderrLineSafe')
|
||||
.mockImplementation(() => undefined);
|
||||
const authorization = failFinalAuthorization(failure);
|
||||
const { server, loaded } = await loadReplacement(true);
|
||||
expect(loaded.status).toBe(409);
|
||||
expect(loaded.body.code).toBe('hosted_turn_recovery_required');
|
||||
expect(acquireSpy).toHaveBeenCalledOnce();
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
expect(
|
||||
stderr.mock.calls.some(
|
||||
([line]) => line.includes('stays owed') && line.includes(PROMPT_ID),
|
||||
),
|
||||
).toBe(true);
|
||||
authorization.mockRestore();
|
||||
const reloaded = await replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
});
|
||||
expect(reloaded.status).toBe(200);
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
await replacementHeaders(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
).expect(204);
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
expect(
|
||||
(release.mock.contexts[0] as HostedWorkspaceBroker).runtimeSessionId,
|
||||
).toBe(PROMPT_ID);
|
||||
},
|
||||
);
|
||||
|
||||
it('re-answers a redriven takeover load whose reply was lost', async () => {
|
||||
await parkToolTurn();
|
||||
const execute = vi
|
||||
@@ -10326,6 +10761,405 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
||||
expect(loaded.body.reason).toBe('model_start');
|
||||
});
|
||||
|
||||
it('carries a resident kernel decline reason onto the wire', async () => {
|
||||
const { server } = await parkToolTurn(true);
|
||||
vi.spyOn(
|
||||
LocalManagedSessionAuthority.prototype,
|
||||
'harnessRunAuthorization',
|
||||
).mockResolvedValue({ status: 'blocked', reason: 'identity_mismatch' });
|
||||
const declined = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({ managedSessionStore: store(), driveRuntimeRecovery: true });
|
||||
expect(declined.status).toBe(409);
|
||||
expect(declined.body.code).toBe('hosted_turn_recovery_declined');
|
||||
expect(declined.body.reason).toBe('checkpoint_blocked');
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
'reattaches a resident requested approval (passive=%s)',
|
||||
async (passive) => {
|
||||
const { server, clientId } = await parkToolTurn(true);
|
||||
mockAuthorizationWithPhase('await_approval', { state: 'requested' });
|
||||
const loaded = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: store(),
|
||||
toolProfile: FILE_PROFILE,
|
||||
[passive ? 'passiveManagedRuntimeRecovery' : 'driveRuntimeRecovery']:
|
||||
true,
|
||||
});
|
||||
expect(loaded.status).toBe(200);
|
||||
expect(loaded.body.clientId).toBe(clientId);
|
||||
expect(loaded.body.recoveryRequired).toBeUndefined();
|
||||
expect(acquireSpy).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it.each([false, true])(
|
||||
'settles a payable resident projection after a lost terminal write (passive=%s)',
|
||||
async (passive) => {
|
||||
await parkToolTurn();
|
||||
vi.mocked(HostedWorkspaceBroker.prototype.execute).mockResolvedValue({
|
||||
executionStatus: 'success',
|
||||
responseParts: [{ text: 'written' }],
|
||||
} as never);
|
||||
const { server, loaded } = await loadReplacement();
|
||||
expect(loaded.status).toBe(200);
|
||||
const recovery = loaded.body._meta['qwen.daemon.managedRuntimeRecovery'];
|
||||
const originalWrite = ManagedSessionRecordSink.prototype.write;
|
||||
const write = vi
|
||||
.spyOn(ManagedSessionRecordSink.prototype, 'write')
|
||||
.mockImplementation(function (this: ManagedSessionRecordSink, item) {
|
||||
if (item.subtype === 'turn_result')
|
||||
return Promise.reject(new Error('terminal write unavailable'));
|
||||
return originalWrite.call(this, item);
|
||||
});
|
||||
await replacementHeaders(
|
||||
supertest(server).post(
|
||||
`/session/${SESSION_ID}/managed-runtime/continue`,
|
||||
),
|
||||
)
|
||||
.set('X-Qwen-Client-Id', loaded.body.clientId)
|
||||
.send({
|
||||
promptId: PROMPT_ID,
|
||||
checkpointId: recovery.checkpointId,
|
||||
activationId: recovery.activationId,
|
||||
})
|
||||
.expect(200);
|
||||
await vi.waitFor(async () => {
|
||||
const status = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/status`),
|
||||
).set('X-Qwen-Client-Id', loaded.body.clientId);
|
||||
expect(status.body.hasActivePrompt).toBe(false);
|
||||
expect(status.body.recoveryBlocked).toBe(true);
|
||||
});
|
||||
write.mockRestore();
|
||||
const history = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/files/history`),
|
||||
).set('X-Qwen-Client-Id', loaded.body.clientId);
|
||||
expect(history.body.history.pendingTurn).toBeNull();
|
||||
expect(history.body.history.pendingUndo).toBeNull();
|
||||
mockAuthorizationWithPhase('turn_settled', null);
|
||||
const redriven = await replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
[passive ? 'passiveManagedRuntimeRecovery' : 'driveRuntimeRecovery']:
|
||||
true,
|
||||
});
|
||||
expect(redriven.status).toBe(200);
|
||||
expect(redriven.body.clientId).toBe(loaded.body.clientId);
|
||||
await vi.waitFor(async () => {
|
||||
const transcript = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/transcript`),
|
||||
).set('X-Qwen-Client-Id', loaded.body.clientId);
|
||||
expect(transcript.body.events).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
type: 'turn_complete',
|
||||
promptId: PROMPT_ID,
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it('refuses a resident inapplicable redrive while a rewind owns the prompt slot', async () => {
|
||||
const realAuthorization =
|
||||
LocalManagedSessionAuthority.prototype.harnessRunAuthorization;
|
||||
await parkToolTurn();
|
||||
vi.mocked(HostedWorkspaceBroker.prototype.execute).mockResolvedValue({
|
||||
executionStatus: 'success',
|
||||
responseParts: [{ text: 'written' }],
|
||||
} as never);
|
||||
const { server, loaded } = await loadReplacement();
|
||||
const recovery = loaded.body._meta['qwen.daemon.managedRuntimeRecovery'];
|
||||
const originalWrite = ManagedSessionRecordSink.prototype.write;
|
||||
const write = vi
|
||||
.spyOn(ManagedSessionRecordSink.prototype, 'write')
|
||||
.mockImplementation(function (this: ManagedSessionRecordSink, item) {
|
||||
if (item.subtype === 'turn_result')
|
||||
return Promise.reject(new Error('terminal write unavailable'));
|
||||
return originalWrite.call(this, item);
|
||||
});
|
||||
await replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/managed-runtime/continue`),
|
||||
)
|
||||
.set('X-Qwen-Client-Id', loaded.body.clientId)
|
||||
.send({
|
||||
promptId: PROMPT_ID,
|
||||
checkpointId: recovery.checkpointId,
|
||||
activationId: recovery.activationId,
|
||||
})
|
||||
.expect(200);
|
||||
await vi.waitFor(async () => {
|
||||
const status = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/status`),
|
||||
).set('X-Qwen-Client-Id', loaded.body.clientId);
|
||||
expect(status.body.hasActivePrompt).toBe(false);
|
||||
expect(status.body.recoveryBlocked).toBe(true);
|
||||
});
|
||||
write.mockRestore();
|
||||
// An approval-pending redrive answers the inapplicable resident without
|
||||
// a projection, clearing the resident latch.
|
||||
mockAuthorizationWithPhase('await_approval', { state: 'requested' });
|
||||
const armed = await replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
driveRuntimeRecovery: true,
|
||||
});
|
||||
expect(armed.status).toBe(200);
|
||||
// The next redrive turns payable: hold it inside
|
||||
// settleProjectablePromptId, admit a rewind inside the window, and
|
||||
// only then release the held read.
|
||||
let resumeSettle!: () => void;
|
||||
const settleHeld = new Promise<void>((resolve) => {
|
||||
resumeSettle = resolve;
|
||||
});
|
||||
let settleParked = false;
|
||||
let helperAnswered = false;
|
||||
vi.mocked(
|
||||
LocalManagedSessionAuthority.prototype.harnessRunAuthorization,
|
||||
).mockImplementation(async function (this: LocalManagedSessionAuthority) {
|
||||
if (helperAnswered && !settleParked) {
|
||||
settleParked = true;
|
||||
await settleHeld;
|
||||
}
|
||||
const authorization = await realAuthorization.call(this);
|
||||
helperAnswered = true;
|
||||
if (authorization.status !== 'runnable') return authorization;
|
||||
return {
|
||||
...authorization,
|
||||
checkpoint: {
|
||||
...authorization.checkpoint,
|
||||
continuation: {
|
||||
...authorization.checkpoint.continuation,
|
||||
phase: 'turn_settled',
|
||||
},
|
||||
approval: null,
|
||||
},
|
||||
} as never;
|
||||
});
|
||||
const redriven = replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
)
|
||||
.send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
driveRuntimeRecovery: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
await vi.waitFor(() => {
|
||||
expect(settleParked).toBe(true);
|
||||
});
|
||||
// The rewind's own history read is the only resource read in flight
|
||||
// now, so gating it parks the rewind after it took the prompt slot.
|
||||
let resumeRewindRead!: () => void;
|
||||
const rewindReadHeld = new Promise<void>((resolve) => {
|
||||
resumeRewindRead = resolve;
|
||||
});
|
||||
let rewindParked = false;
|
||||
const realRead = LocalManagedSessionResourceStore.prototype.read;
|
||||
const readSpy = vi
|
||||
.spyOn(LocalManagedSessionResourceStore.prototype, 'read')
|
||||
.mockImplementation(async function (
|
||||
this: LocalManagedSessionResourceStore,
|
||||
...args: Parameters<LocalManagedSessionResourceStore['read']>
|
||||
) {
|
||||
if (!rewindParked) {
|
||||
rewindParked = true;
|
||||
await rewindReadHeld;
|
||||
}
|
||||
return realRead.apply(this, args);
|
||||
} as never);
|
||||
const rewind = replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/files/rewind`),
|
||||
)
|
||||
.set('X-Qwen-Client-Id', armed.body.clientId)
|
||||
.send({ requestId: randomUUID(), promptId: PROMPT_ID })
|
||||
.then((response) => response);
|
||||
await vi.waitFor(() => {
|
||||
expect(rewindParked).toBe(true);
|
||||
});
|
||||
resumeSettle();
|
||||
const refused = await redriven;
|
||||
expect(refused.status).toBe(409);
|
||||
expect(refused.body.code).toBe('hosted_turn_active');
|
||||
// The rewind must keep its prompt slot: /status may never report the
|
||||
// Session idle while a workspace mutation is in flight.
|
||||
const during = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/status`),
|
||||
).set('X-Qwen-Client-Id', armed.body.clientId);
|
||||
expect(during.body.hasActivePrompt).toBe(true);
|
||||
resumeRewindRead();
|
||||
readSpy.mockRestore();
|
||||
await rewind;
|
||||
await vi.waitFor(async () => {
|
||||
const status = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/status`),
|
||||
).set('X-Qwen-Client-Id', armed.body.clientId);
|
||||
expect(status.body.hasActivePrompt).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
it.each(['removed', 'closing'] as const)(
|
||||
'refuses a resident inapplicable redrive when its attachment is %s',
|
||||
async (attachmentState) => {
|
||||
await parkToolTurn();
|
||||
vi.mocked(HostedWorkspaceBroker.prototype.execute).mockResolvedValue({
|
||||
executionStatus: 'success',
|
||||
responseParts: [{ text: 'written' }],
|
||||
} as never);
|
||||
const { server, loaded } = await loadReplacement();
|
||||
expect(loaded.status).toBe(200);
|
||||
const originalAuthorization =
|
||||
LocalManagedSessionAuthority.prototype.harnessRunAuthorization;
|
||||
let resumeAuthorization!: () => void;
|
||||
const authorizationGate = new Promise<void>((resolve) => {
|
||||
resumeAuthorization = resolve;
|
||||
});
|
||||
let authorizationCalls = 0;
|
||||
let authorizationHeld = false;
|
||||
vi.spyOn(
|
||||
LocalManagedSessionAuthority.prototype,
|
||||
'harnessRunAuthorization',
|
||||
).mockImplementation(async function (this: LocalManagedSessionAuthority) {
|
||||
const authorization = await originalAuthorization.call(this);
|
||||
if (authorization.status !== 'runnable') return authorization;
|
||||
if (++authorizationCalls === 2) {
|
||||
authorizationHeld = true;
|
||||
await authorizationGate;
|
||||
}
|
||||
return {
|
||||
...authorization,
|
||||
checkpoint: {
|
||||
...authorization.checkpoint,
|
||||
continuation: {
|
||||
...authorization.checkpoint.continuation,
|
||||
phase: 'await_approval',
|
||||
},
|
||||
approval: { state: 'requested' },
|
||||
},
|
||||
} as never;
|
||||
});
|
||||
let resumeClose!: () => void;
|
||||
const closeGate = new Promise<void>((resolve) => {
|
||||
resumeClose = resolve;
|
||||
});
|
||||
let closeHeld = false;
|
||||
if (attachmentState === 'closing') {
|
||||
vi.mocked(
|
||||
HostedWorkspaceBroker.prototype.release,
|
||||
).mockImplementationOnce(async () => {
|
||||
closeHeld = true;
|
||||
await closeGate;
|
||||
});
|
||||
}
|
||||
const load = replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
)
|
||||
.send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
driveRuntimeRecovery: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
let close: Promise<supertest.Response> | undefined;
|
||||
try {
|
||||
await vi.waitFor(() => expect(authorizationHeld).toBe(true));
|
||||
close = replacementHeaders(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
).then((response) => response);
|
||||
if (attachmentState === 'closing') {
|
||||
await vi.waitFor(() => expect(closeHeld).toBe(true));
|
||||
} else {
|
||||
expect((await close).status).toBe(204);
|
||||
}
|
||||
resumeAuthorization();
|
||||
const refused = await load;
|
||||
expect(refused.status).toBe(attachmentState === 'removed' ? 404 : 409);
|
||||
expect(refused.body.code).toBe(
|
||||
attachmentState === 'removed'
|
||||
? 'hosted_session_not_found'
|
||||
: 'hosted_session_closing',
|
||||
);
|
||||
} finally {
|
||||
resumeAuthorization();
|
||||
resumeClose();
|
||||
await Promise.allSettled(close ? [load, close] : [load]);
|
||||
}
|
||||
expect((await close!).status).toBe(204);
|
||||
},
|
||||
);
|
||||
|
||||
it('refuses a cancellation takeover attachment deleted during settlement', async () => {
|
||||
const { server } = await parkToolTurn(true);
|
||||
const authorize =
|
||||
LocalManagedSessionAuthority.prototype.harnessRunAuthorization;
|
||||
vi.spyOn(
|
||||
LocalManagedSessionAuthority.prototype,
|
||||
'harnessRunAuthorization',
|
||||
).mockImplementation(async function (this: LocalManagedSessionAuthority) {
|
||||
const result = await authorize.call(this);
|
||||
if (result.status !== 'runnable') return result;
|
||||
return {
|
||||
...result,
|
||||
checkpoint: {
|
||||
...result.checkpoint,
|
||||
approval: null,
|
||||
tools: {
|
||||
...result.checkpoint.tools,
|
||||
items: result.checkpoint.tools!.items.map((item) => ({
|
||||
...item,
|
||||
state: 'settled',
|
||||
consumed: true,
|
||||
})),
|
||||
},
|
||||
},
|
||||
} as never;
|
||||
});
|
||||
let started!: () => void;
|
||||
const settling = new Promise<void>((resolve) => {
|
||||
started = resolve;
|
||||
});
|
||||
let resume!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
resume = resolve;
|
||||
});
|
||||
const originalWrite = ManagedSessionRecordSink.prototype.write;
|
||||
vi.spyOn(ManagedSessionRecordSink.prototype, 'write').mockImplementation(
|
||||
async function (this: ManagedSessionRecordSink, item) {
|
||||
await originalWrite.call(this, item);
|
||||
if (item.subtype === 'turn_result') {
|
||||
started();
|
||||
await held;
|
||||
}
|
||||
},
|
||||
);
|
||||
const load = headers(supertest(server).post(`/session/${SESSION_ID}/load`))
|
||||
.send({
|
||||
managedSessionStore: store(),
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
cancellationTakeover: true,
|
||||
})
|
||||
.then((response) => response);
|
||||
await settling;
|
||||
try {
|
||||
await headers(supertest(server).delete(`/session/${SESSION_ID}`)).expect(
|
||||
204,
|
||||
);
|
||||
} finally {
|
||||
resume();
|
||||
}
|
||||
const refused = await load;
|
||||
expect(refused.status).toBe(404);
|
||||
expect(refused.body.code).toBe('hosted_session_not_found');
|
||||
});
|
||||
|
||||
it('re-answers the unchanged recovery after a lost cancellation report', async () => {
|
||||
// Nothing consumed means nothing to unwedge: a failed cancel cannot
|
||||
// break the re-answer — the redrive recomputes from the attached state
|
||||
@@ -11050,11 +11884,22 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
||||
.mockResolvedValue();
|
||||
const { server, loaded } = await loadReplacement(true);
|
||||
expect(loaded.status).toBe(200);
|
||||
// The cancellation path adopts the Runtime Session; it still dispatches nothing.
|
||||
expect(acquireSpy).toHaveBeenCalled();
|
||||
// The adoption is held for the cancel route: loading never releases it.
|
||||
// Passive takeover adopts the original Runtime without dispatching work.
|
||||
expect(acquireSpy).toHaveBeenCalledTimes(1);
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
const recovery = loaded.body._meta?.[
|
||||
// The owner may lose its first load reply after Harness registered it.
|
||||
const reloaded = await replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/load`),
|
||||
).send({
|
||||
managedSessionStore: storeFor(BOOT_ID_2),
|
||||
toolProfile: FILE_PROFILE,
|
||||
passiveManagedRuntimeRecovery: true,
|
||||
});
|
||||
expect(reloaded.status).toBe(200);
|
||||
expect(reloaded.body.clientId).toBe(loaded.body.clientId);
|
||||
expect(acquireSpy).toHaveBeenCalledTimes(2);
|
||||
expect(release).not.toHaveBeenCalled();
|
||||
const recovery = reloaded.body._meta?.[
|
||||
'qwen.daemon.managedRuntimeRecovery'
|
||||
] as {
|
||||
phase: string;
|
||||
|
||||
@@ -182,6 +182,8 @@ function settledTurnOutcome(abort: AbortController): {
|
||||
|
||||
interface HostedSession {
|
||||
managed: ManagedSession;
|
||||
storeBaseUrl: string;
|
||||
definition: Record<string, unknown> | null;
|
||||
clientId: string;
|
||||
cwd: string;
|
||||
streams: Set<() => void>;
|
||||
@@ -203,7 +205,7 @@ interface HostedSession {
|
||||
hooksBusy?: boolean;
|
||||
mcpBusy?: boolean;
|
||||
mcpClosing?: boolean;
|
||||
mcpRecovering?: boolean;
|
||||
mcpRecovering: number;
|
||||
approval?: HostedApprovalSettings;
|
||||
waiters: HostedApprovalWaiters;
|
||||
stores?: HttpManagedSessionStores;
|
||||
@@ -865,6 +867,7 @@ async function settleProjectablePromptId(
|
||||
return null;
|
||||
if (
|
||||
fileHistory &&
|
||||
(fileHistory.pendingTurn || fileHistory.pendingUndo) &&
|
||||
!(await canSettleHostedFileHistory(managed, {
|
||||
...fileHistory,
|
||||
pendingTurn: promptId,
|
||||
@@ -1807,6 +1810,78 @@ export function registerHostedHarnessSessionRoutes(
|
||||
const opening = new Set<string>();
|
||||
const epoch = contract.bootId.replaceAll('-', '_');
|
||||
|
||||
const sendAttachment = (
|
||||
res: Response,
|
||||
sessionId: string,
|
||||
session: HostedSession,
|
||||
recovery?: HostedRuntimeRecoveryReport,
|
||||
): void => {
|
||||
res.status(200).json({
|
||||
sessionId,
|
||||
clientId: session.clientId,
|
||||
workspaceCwd: session.cwd,
|
||||
lastEventId: session.managed.authority.committedSequence,
|
||||
eventEpoch: epoch,
|
||||
// A Harness older than approvals omits this, so a caller can tell.
|
||||
...(session.approval ? { approvalMode: session.approval.mode } : {}),
|
||||
...(session.blocked || session.hooks?.hasPendingOperations
|
||||
? { recoveryRequired: true }
|
||||
: {}),
|
||||
...(recovery
|
||||
? { _meta: { 'qwen.daemon.managedRuntimeRecovery': recovery } }
|
||||
: {}),
|
||||
});
|
||||
};
|
||||
|
||||
const answerResidentInapplicable = async (
|
||||
res: Response,
|
||||
sessionId: string,
|
||||
resident: HostedSession,
|
||||
promptId: string,
|
||||
): Promise<void> => {
|
||||
const authorization =
|
||||
await resident.managed.authority.harnessRunAuthorization();
|
||||
const approvalPending =
|
||||
authorization.status === 'runnable' &&
|
||||
authorization.checkpoint.approval?.state === 'requested';
|
||||
const settle = approvalPending
|
||||
? null
|
||||
: await settleProjectablePromptId(
|
||||
resident.managed,
|
||||
resident,
|
||||
await readHostedFileHistory(resident.managed),
|
||||
promptId,
|
||||
);
|
||||
if (!approvalPending && settle === null) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Session ${sessionId} redrive refused (takeover_inapplicable_unpayable): prompt=${promptId}`,
|
||||
);
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
if (sessions.get(sessionId) !== resident) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 404, 'hosted_session_not_found');
|
||||
return;
|
||||
}
|
||||
if (resident.mcpClosing) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 409, 'hosted_session_closing');
|
||||
return;
|
||||
}
|
||||
if (resident.active !== undefined) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 409, 'hosted_turn_active');
|
||||
return;
|
||||
}
|
||||
refusedAdoptions.delete(sessionId);
|
||||
resident.blocked = false;
|
||||
sendAttachment(res, sessionId, resident);
|
||||
if (settle !== null)
|
||||
runSettleProjection(resident, sessionId, settle, brokerOptions);
|
||||
};
|
||||
|
||||
const open = async (
|
||||
req: Request,
|
||||
res: Response,
|
||||
@@ -1909,227 +1984,334 @@ export function registerHostedHarnessSessionRoutes(
|
||||
error(res, 400, 'invalid_hosted_lifecycle_authority');
|
||||
return;
|
||||
}
|
||||
// Cold loads stay inert: only an explicit takeover request may touch
|
||||
// the Broker or settle anything.
|
||||
const takeoverFlags =
|
||||
body?.['passiveManagedRuntimeRecovery'] === true ||
|
||||
body?.['driveRuntimeRecovery'] === true;
|
||||
// One shape for every successful open/load answer — a takeover load
|
||||
// redriven after a lost reply must be indistinguishable from the answer
|
||||
// it replaces, so all three sites build it here.
|
||||
const attachmentReply = (
|
||||
session: HostedSession,
|
||||
recovery?: HostedRuntimeRecoveryReport,
|
||||
) => ({
|
||||
sessionId,
|
||||
clientId: session.clientId,
|
||||
workspaceCwd: cwd,
|
||||
lastEventId: session.managed.authority.committedSequence,
|
||||
eventEpoch: epoch,
|
||||
...(session.approval ? { approvalMode: session.approval.mode } : {}),
|
||||
...(session.blocked || session.hooks?.hasPendingOperations
|
||||
? { recoveryRequired: true }
|
||||
: {}),
|
||||
...(recovery
|
||||
? { _meta: { 'qwen.daemon.managedRuntimeRecovery': recovery } }
|
||||
: {}),
|
||||
});
|
||||
const attached = sessions.get(sessionId);
|
||||
if (attached !== undefined && lifecycle) {
|
||||
const resident = sessions.get(sessionId);
|
||||
const passiveRecovery = body?.['passiveManagedRuntimeRecovery'] === true;
|
||||
const driveRecovery = body?.['driveRuntimeRecovery'] === true;
|
||||
const takeoverFlags = passiveRecovery || driveRecovery;
|
||||
if (resident !== undefined && lifecycle) {
|
||||
if (
|
||||
!isDeepStrictEqual(store, attached.storeDescriptor) ||
|
||||
!isDeepStrictEqual(store, resident.storeDescriptor) ||
|
||||
toolProfile !== HOSTED_WORKSPACE_FILE_PROFILE ||
|
||||
attached.toolProfile !== HOSTED_WORKSPACE_FILE_PROFILE ||
|
||||
(attached.lifecycle &&
|
||||
attached.lifecycle.operationId !== lifecycle.operationId)
|
||||
resident.toolProfile !== HOSTED_WORKSPACE_FILE_PROFILE ||
|
||||
(resident.lifecycle &&
|
||||
resident.lifecycle.operationId !== lifecycle.operationId)
|
||||
) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
if (
|
||||
attached.active ||
|
||||
attached.mcpBusy ||
|
||||
attached.mcpRecovering ||
|
||||
attached.hooksBusy
|
||||
resident.active ||
|
||||
resident.mcpBusy ||
|
||||
resident.mcpRecovering ||
|
||||
resident.hooksBusy
|
||||
) {
|
||||
error(res, 409, 'hosted_turn_active');
|
||||
return;
|
||||
}
|
||||
const previous = attached.lifecycle;
|
||||
attached.hooksBusy = true;
|
||||
attached.stores!.setLifecycleAuthority(lifecycle);
|
||||
const previous = resident.lifecycle;
|
||||
resident.hooksBusy = true;
|
||||
resident.stores!.setLifecycleAuthority(lifecycle);
|
||||
try {
|
||||
await attached.stores!.assertWritable();
|
||||
attached.lifecycle = lifecycle;
|
||||
res.status(200).json(attachmentReply(attached));
|
||||
await resident.stores!.assertWritable();
|
||||
resident.lifecycle = lifecycle;
|
||||
sendAttachment(res, sessionId, resident);
|
||||
} catch (cause) {
|
||||
attached.stores!.setLifecycleAuthority(previous);
|
||||
resident.stores!.setLifecycleAuthority(previous);
|
||||
debugLogger.warn('Hosted lifecycle attachment claim rejected:', cause);
|
||||
error(res, 503, 'managed_session_open_failed');
|
||||
} finally {
|
||||
attached.hooksBusy = false;
|
||||
resident.hooksBusy = false;
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (attached !== undefined && !create) {
|
||||
const passive = body?.['passiveManagedRuntimeRecovery'] === true;
|
||||
if (!attached.hooks && takeoverFlags) {
|
||||
// The re-answer hands over the attached Session's client id again;
|
||||
// it must also re-prove the store identity the attachment was
|
||||
// opened with — the writer fence proves the generation, this proves
|
||||
// the tenant/workspace the caller claims to continue for.
|
||||
const attachedKey = attached.managed.authority.sessionHeader.sessionKey;
|
||||
// The re-answer hands over the resident Session's client id again, so
|
||||
// both below branches re-prove the store identity the attachment was
|
||||
// opened with once, before either runs: the writer fence proves the
|
||||
// generation, this proves the tenant/Workspace the caller claims to
|
||||
// continue for. A drifted Session Store address is the caller's own
|
||||
// lookup failing, so it answers a retryable conflict with its own code.
|
||||
if (resident !== undefined && !create) {
|
||||
const key = resident.managed.authority.sessionHeader.sessionKey;
|
||||
if (
|
||||
key.tenantId !== store.tenantId ||
|
||||
key.workspaceId !== store.workspaceId
|
||||
) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
if (resident.storeBaseUrl !== store.baseUrl) {
|
||||
error(res, 409, 'hosted_session_store_mismatch');
|
||||
return;
|
||||
}
|
||||
// The cancellation signal pays identically on an attached re-answer
|
||||
// (R9-2): the settle separation lived only on the first-load branch,
|
||||
// so a cancellation takeover forced onto an attached Session fell
|
||||
// into the passive kernel — which never advances a durable wait
|
||||
// passively, and threw the park back as an unknown phase once the
|
||||
// wait had ended since the attach. The no-tool arm settles
|
||||
// unconditionally here too, and the owed-work gate reads the work
|
||||
// exactly as on the first load.
|
||||
if (
|
||||
!resident.hooks &&
|
||||
resident.active === undefined &&
|
||||
(passiveRecovery || driveRecovery) &&
|
||||
body?.['cancellationTakeover'] === true
|
||||
) {
|
||||
const parkedForCancellation = unsettledPromptId(resident);
|
||||
if (
|
||||
store.tenantId !== attachedKey.tenantId ||
|
||||
store.workspaceId !== attachedKey.workspaceId
|
||||
parkedForCancellation !== undefined &&
|
||||
(resident.toolProfile === undefined ||
|
||||
!brokerOptions ||
|
||||
(await parkNeedsNoRuntimeSettlement(resident, resident.managed)))
|
||||
) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
// A takeover load whose reply was lost is redriven against the
|
||||
// Session it already attached — no continue/cancel can have been
|
||||
// admitted since, because their identities ride the lost reply, so
|
||||
// the attached state is still exactly what the first load left
|
||||
// behind. Answer again from that state instead of retaining a
|
||||
// snapshot — nothing is consumed, so a lost cancel report cannot
|
||||
// wedge the re-answer either (D6). The re-run is read-only apart
|
||||
// from the Broker acquire, which is idempotent under the same
|
||||
// Runtime Session identity.
|
||||
const parked = unsettledPromptId(attached);
|
||||
if (parked !== undefined) {
|
||||
if (attached.active !== undefined) {
|
||||
if (resident.active !== undefined) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
// The cancellation signal pays identically on an attached
|
||||
// re-answer (R9-2): the settle separation lived only on the
|
||||
// first-load branch, so a cancellation takeover forced onto an
|
||||
// attached Session fell into the passive kernel — which never
|
||||
// advances a durable wait passively, and threw the park back as
|
||||
// an unknown phase once the wait had ended since the attach.
|
||||
// The no-tool arm settles unconditionally here too, and the
|
||||
// owed-work gate reads the work exactly as on the first load.
|
||||
if (
|
||||
body?.['cancellationTakeover'] === true &&
|
||||
(attached.toolProfile === undefined ||
|
||||
!brokerOptions ||
|
||||
(await parkNeedsNoRuntimeSettlement(attached, attached.managed)))
|
||||
) {
|
||||
try {
|
||||
await settleCancelledHarnessTurn(
|
||||
attached.managed,
|
||||
attached,
|
||||
sessionId,
|
||||
parked,
|
||||
);
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Session ${sessionId} settles the cancelled park on the redriven load: prompt=${parked}`,
|
||||
);
|
||||
// The journal owes nothing more: the re-answer carries the
|
||||
// refreshed watermark the already-running stream settles
|
||||
// from.
|
||||
res.status(200).json(attachmentReply(attached));
|
||||
return;
|
||||
} catch (cause) {
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Session ${sessionId} redrive refused (takeover_unavailable): profile=${attached.toolProfile ?? 'none'} broker=${brokerOptions ? 'ready' : 'none'} settle=${String(cause)}`,
|
||||
);
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (attached.toolProfile === undefined || !brokerOptions) {
|
||||
if (!passive) {
|
||||
recoveryDeclined(res, 'model_start');
|
||||
return;
|
||||
}
|
||||
// No kernel is consulted on this arm — recoverHostedRuntimeTurn
|
||||
// is only reachable with a tool profile and broker options — so
|
||||
// there is no inapplicable to mirror. The cancellation load
|
||||
// keeps the first load's retriable refusal: a plain attach
|
||||
// would let the coordinator's cancel land on a Session with no
|
||||
// live Turn (the plain cancel route only aborts session.active),
|
||||
// suppressing every other settle path while the journal keeps
|
||||
// the input unsettled — a permanent wedge (R10-1).
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const outcome = await recoverHostedRuntimeTurn({
|
||||
session: attached.managed,
|
||||
await settleCancelledHarnessTurn(
|
||||
resident.managed,
|
||||
resident,
|
||||
sessionId,
|
||||
cwd,
|
||||
promptId: parked,
|
||||
brokerOptions,
|
||||
passive,
|
||||
leaseAlreadyHeld: attached.runtimeLeaseHeld !== undefined,
|
||||
});
|
||||
if (outcome.kind === 'declined') {
|
||||
recoveryDeclined(res, outcome.reason);
|
||||
parkedForCancellation,
|
||||
);
|
||||
if (sessions.get(sessionId) !== resident) {
|
||||
error(res, 404, 'hosted_session_not_found');
|
||||
return;
|
||||
}
|
||||
if (outcome.kind === 'inapplicable') {
|
||||
// R11-2: a requested approval keeps the plain attach (the
|
||||
// resolve route writes the decision durably); turn_settled
|
||||
// must be settled HERE — this arm had no settle block at
|
||||
// all, so the missing terminal record is projected inline —
|
||||
// or the redrive keeps the retriable refusal when the
|
||||
// projection cannot pay either.
|
||||
const inapplicableVerdict = await attached.managed.authority
|
||||
.harnessRunAuthorization()
|
||||
.catch(() => undefined);
|
||||
const approvalPending =
|
||||
inapplicableVerdict?.status === 'runnable' &&
|
||||
inapplicableVerdict.checkpoint.approval?.state === 'requested';
|
||||
if (approvalPending) {
|
||||
res.status(200).json(attachmentReply(attached));
|
||||
return;
|
||||
}
|
||||
const inapplicableFileHistory =
|
||||
(await readHostedFileHistory(attached.managed)) ?? null;
|
||||
const settle = await settleProjectablePromptId(
|
||||
attached.managed,
|
||||
attached,
|
||||
inapplicableFileHistory,
|
||||
parked,
|
||||
);
|
||||
if (settle === null) {
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Session ${sessionId} redrive refused (takeover_inapplicable_unpayable): prompt=${parked}`,
|
||||
);
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
res.status(200).json(attachmentReply(attached));
|
||||
runSettleProjection(attached, sessionId, settle, brokerOptions);
|
||||
if (resident.mcpClosing) {
|
||||
error(res, 409, 'hosted_session_closing');
|
||||
return;
|
||||
}
|
||||
const recovery = outcome.turn.report;
|
||||
if (outcome.turn.acquiredRuntime)
|
||||
attached.runtimeLeaseHeld =
|
||||
outcome.turn.report.executions[0]?.runtimeSessionId ??
|
||||
outcome.turn.promptId;
|
||||
res.status(200).json(attachmentReply(attached, recovery));
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Session ${sessionId} settles the cancelled park on the redriven load: prompt=${parkedForCancellation}`,
|
||||
);
|
||||
// The journal owes nothing more: the re-answer carries the
|
||||
// refreshed watermark the already-running stream settles from.
|
||||
refusedAdoptions.delete(sessionId);
|
||||
sendAttachment(res, sessionId, resident);
|
||||
return;
|
||||
} catch (cause) {
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Harness recovery re-answer of session ${sessionId} failed: ${String(cause)}`,
|
||||
`qwen serve: Hosted Session ${sessionId} redrive refused (takeover_unavailable): profile=${resident.toolProfile ?? 'none'} broker=${brokerOptions ? 'ready' : 'none'} settle=${String(cause)}`,
|
||||
);
|
||||
// Retry-inviting, like the first load's recovery failure; the
|
||||
// attached Session keeps its owed lease for the next redrive.
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (
|
||||
opening.has(sessionId) ||
|
||||
(resident && (create || (!passiveRecovery && !driveRecovery)))
|
||||
) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
// A continuation load redriven after a lost reply is answered from the
|
||||
// Session it already attached: the writer fence above proves this load
|
||||
// targets this generation, and no continue/cancel can have been admitted
|
||||
// since (its identities ride the lost reply), so the resident state is
|
||||
// still exactly what the first load left behind. Answer again from that
|
||||
// state instead of wedging the Turn on a 409 loop; the re-run is read-only
|
||||
// apart from the Broker acquire, which is idempotent under the same
|
||||
// Runtime Session identity. A passive load takes the stricter resident
|
||||
// path below, which validates the store and the tool profile first.
|
||||
if (resident !== undefined && driveRecovery && !passiveRecovery) {
|
||||
if (resident.hooks) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
const parked = unsettledPromptId(resident);
|
||||
if (parked === undefined) {
|
||||
// No single parked Turn: the first load's answer still holds, so the
|
||||
// redrive gets the same attachment restated — including a blocked
|
||||
// Session, whose recoveryRequired the coordinator already handles.
|
||||
res.status(200).json(attachmentReply(attached));
|
||||
refusedAdoptions.delete(sessionId);
|
||||
sendAttachment(res, sessionId, resident);
|
||||
return;
|
||||
}
|
||||
if (resident.active !== undefined) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
return;
|
||||
}
|
||||
if (resident.toolProfile === undefined || !brokerOptions) {
|
||||
recoveryDeclined(res, 'model_start');
|
||||
return;
|
||||
}
|
||||
let recovery: HostedRuntimeRecoveryReport;
|
||||
try {
|
||||
const outcome = await recoverHostedRuntimeTurn({
|
||||
session: resident.managed,
|
||||
sessionId,
|
||||
cwd,
|
||||
promptId: parked,
|
||||
brokerOptions,
|
||||
passive: false,
|
||||
leaseAlreadyHeld: resident.runtimeLeaseHeld !== undefined,
|
||||
});
|
||||
if (outcome.kind === 'declined') {
|
||||
recoveryDeclined(res, outcome.reason);
|
||||
return;
|
||||
}
|
||||
if (outcome.kind === 'inapplicable') {
|
||||
await answerResidentInapplicable(res, sessionId, resident, parked);
|
||||
return;
|
||||
}
|
||||
recovery = outcome.turn.report;
|
||||
if (outcome.turn.acquiredRuntime)
|
||||
resident.runtimeLeaseHeld =
|
||||
outcome.turn.report.executions[0]?.runtimeSessionId ??
|
||||
outcome.turn.promptId;
|
||||
} catch (cause) {
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Harness recovery re-answer of session ${sessionId} failed: ${String(cause)}`,
|
||||
);
|
||||
// Retry-inviting, like the first load's recovery failure; the
|
||||
// attached Session keeps its owed lease for the next redrive.
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
// A teardown admitted during the await above released nothing (the
|
||||
// lease was not recorded yet) and left no route to hand it back.
|
||||
if (sessions.get(sessionId) !== resident) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 404, 'hosted_session_not_found');
|
||||
return;
|
||||
}
|
||||
if (resident.mcpClosing) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 409, 'hosted_session_closing');
|
||||
return;
|
||||
}
|
||||
refusedAdoptions.delete(sessionId);
|
||||
sendAttachment(res, sessionId, resident, recovery);
|
||||
return;
|
||||
}
|
||||
if (attached !== undefined || opening.has(sessionId)) {
|
||||
error(res, 409, 'hosted_session_already_attached');
|
||||
if (resident) {
|
||||
if (
|
||||
toolProfile === undefined &&
|
||||
isHostedWorkspaceProfile(resident.toolProfile)
|
||||
)
|
||||
toolProfile = resident.toolProfile;
|
||||
const definition = resident.definition;
|
||||
if (
|
||||
definition?.['toolProfile'] !== toolProfile ||
|
||||
JSON.stringify(definition?.['mcpServers']) !==
|
||||
JSON.stringify(mcpServers) ||
|
||||
!isDeepStrictEqual(
|
||||
definition?.['hookCatalog'],
|
||||
hookCatalog ?? definition?.['hookCatalog'],
|
||||
) ||
|
||||
(isHostedWorkspaceShellProfile(toolProfile) &&
|
||||
captureBytes !== undefined &&
|
||||
definition?.['captureBytes'] !== captureBytes)
|
||||
) {
|
||||
error(res, 409, 'hosted_tool_profile_conflict');
|
||||
return;
|
||||
}
|
||||
try {
|
||||
// Reuse the live owner without reopening its writer or driving work.
|
||||
// A lost passive-load reply must still report a parked Runtime Turn.
|
||||
let recovery: HostedRuntimeRecoveryReport | undefined;
|
||||
const parked = !resident.active && unsettledPromptId(resident);
|
||||
if (resident.mcpClosing) {
|
||||
error(res, 409, 'hosted_session_closing');
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!resident.active &&
|
||||
!parked &&
|
||||
hasUnsettledInput(
|
||||
resident,
|
||||
resident.managed.authority.committedSequence,
|
||||
)
|
||||
) {
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
if (parked) {
|
||||
if (!resident.toolProfile || !brokerOptions || resident.hooks) {
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
// The adoption this await publishes lands on a Session every route
|
||||
// can still reach, so the recovery holds close()'s own fence: a
|
||||
// concurrent teardown would otherwise release a lease that is still
|
||||
// mid-adoption and persist the record RELEASED.
|
||||
resident.mcpRecovering += 1;
|
||||
try {
|
||||
const outcome = await recoverHostedRuntimeTurn({
|
||||
session: resident.managed,
|
||||
sessionId,
|
||||
cwd: resident.cwd,
|
||||
promptId: parked,
|
||||
brokerOptions,
|
||||
passive: true,
|
||||
onPassiveRuntimeAcquired: (runtimeSessionId) => {
|
||||
resident.runtimeLeaseHeld = runtimeSessionId;
|
||||
},
|
||||
});
|
||||
if (outcome.kind === 'recovered') {
|
||||
recovery = outcome.turn.report;
|
||||
} else if (outcome.kind === 'inapplicable') {
|
||||
await answerResidentInapplicable(
|
||||
res,
|
||||
sessionId,
|
||||
resident,
|
||||
parked,
|
||||
);
|
||||
return;
|
||||
} else {
|
||||
// A declined passive load was refused before any adoption, so
|
||||
// nothing is owed; the typed code tells the coordinator the
|
||||
// refusal is terminal.
|
||||
recoveryDeclined(res, outcome.reason);
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!resident.active &&
|
||||
unsettledPromptId(resident) &&
|
||||
(!recovery ||
|
||||
parked !== unsettledPromptId(resident) ||
|
||||
recovery.checkpointId !==
|
||||
resident.managed.authority.latestCheckpoint?.checkpointId ||
|
||||
recovery.activationId !==
|
||||
resident.managed.activation.activationId)
|
||||
) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
return;
|
||||
}
|
||||
} finally {
|
||||
resident.mcpRecovering -= 1;
|
||||
}
|
||||
}
|
||||
// The fence above covers the whole adoption, so these exits answer
|
||||
// only a Session another route already dropped.
|
||||
if (sessions.get(sessionId) !== resident) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 404, 'hosted_session_not_found');
|
||||
return;
|
||||
}
|
||||
if (resident.mcpClosing) {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 409, 'hosted_session_closing');
|
||||
return;
|
||||
}
|
||||
refusedAdoptions.delete(sessionId);
|
||||
sendAttachment(
|
||||
res,
|
||||
sessionId,
|
||||
resident,
|
||||
resident.active || !unsettledPromptId(resident)
|
||||
? undefined
|
||||
: recovery,
|
||||
);
|
||||
} catch {
|
||||
noteOwedAdoption(resident, sessionId);
|
||||
error(res, 409, 'hosted_turn_recovery_required');
|
||||
}
|
||||
return;
|
||||
}
|
||||
const sessionKey = {
|
||||
@@ -2265,11 +2447,14 @@ export function registerHostedHarnessSessionRoutes(
|
||||
}
|
||||
const session: HostedSession = {
|
||||
managed,
|
||||
storeBaseUrl: store.baseUrl,
|
||||
definition,
|
||||
clientId: randomUUID(),
|
||||
cwd,
|
||||
streams: new Set(),
|
||||
admissions: new Map(),
|
||||
blocked: false,
|
||||
mcpRecovering: 0,
|
||||
waiters: new HostedApprovalWaiters(),
|
||||
stores,
|
||||
storeDescriptor: store,
|
||||
@@ -2356,7 +2541,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
const wakeBusy = () =>
|
||||
session.active !== undefined ||
|
||||
session.mcpBusy === true ||
|
||||
session.mcpRecovering === true ||
|
||||
session.mcpRecovering > 0 ||
|
||||
session.hooksBusy === true ||
|
||||
session.mcpClosing === true;
|
||||
const wakeBlocked = () =>
|
||||
@@ -2616,6 +2801,9 @@ export function registerHostedHarnessSessionRoutes(
|
||||
promptId: unsettled,
|
||||
brokerOptions,
|
||||
passive: body?.['passiveManagedRuntimeRecovery'] === true,
|
||||
onPassiveRuntimeAcquired: (runtimeSessionId) => {
|
||||
session.runtimeLeaseHeld = runtimeSessionId;
|
||||
},
|
||||
});
|
||||
if (outcome.kind === 'inapplicable') {
|
||||
// R11-2: inapplicable pays only where a settlement route can.
|
||||
@@ -2641,6 +2829,10 @@ export function registerHostedHarnessSessionRoutes(
|
||||
unsettled,
|
||||
);
|
||||
if (settle === null) {
|
||||
// The adoption this load took stays owed: a release here
|
||||
// would persist RELEASED and wedge every retried acquire
|
||||
// of the identity, so the teardown's release pays it.
|
||||
noteOwedAdoption(session, sessionId);
|
||||
await managed.close();
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Session ${sessionId} load refused (takeover_inapplicable_unpayable): prompt=${unsettled}`,
|
||||
@@ -2669,6 +2861,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
// continues as the plain attach it was before G3, so a requested
|
||||
// approval or a cancellation-only load meets its own path.
|
||||
} catch (cause) {
|
||||
noteOwedAdoption(session, sessionId);
|
||||
await managed.close();
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted Harness recovery of session ${sessionId} failed: ${String(cause)}`,
|
||||
@@ -2873,9 +3066,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
// The registered Session now carries the owed lease itself; the
|
||||
// refusal-time record is discharged.
|
||||
refusedAdoptions.delete(sessionId);
|
||||
// A Harness older than approvals omits approvalMode, so a caller can
|
||||
// tell.
|
||||
res.status(200).json(attachmentReply(session, recovery));
|
||||
sendAttachment(res, sessionId, session, recovery);
|
||||
if (settlePromptId)
|
||||
runSettleProjection(session, sessionId, settlePromptId, brokerOptions);
|
||||
} catch (cause) {
|
||||
@@ -3604,7 +3795,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
if (!session.mcp) return error(res, 409, 'hosted_mcp_unavailable');
|
||||
if (session.mcpClosing || session.mcpRecovering)
|
||||
return error(res, 409, 'hosted_mcp_operation_active');
|
||||
session.mcpRecovering = true;
|
||||
session.mcpRecovering += 1;
|
||||
void session.mcp
|
||||
.cancel(req.params['operationId'])
|
||||
.then(
|
||||
@@ -3612,7 +3803,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
() => error(res, 503, 'hosted_mcp_cancel_failed'),
|
||||
)
|
||||
.finally(() => {
|
||||
session.mcpRecovering = false;
|
||||
session.mcpRecovering -= 1;
|
||||
});
|
||||
});
|
||||
|
||||
@@ -3622,7 +3813,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
if (!session.mcp) return error(res, 409, 'hosted_mcp_unavailable');
|
||||
if (session.mcpClosing || session.mcpRecovering)
|
||||
return error(res, 409, 'hosted_mcp_operation_active');
|
||||
session.mcpRecovering = true;
|
||||
session.mcpRecovering += 1;
|
||||
void session.mcp
|
||||
.status(req.params['operationId'])
|
||||
.then(
|
||||
@@ -3630,7 +3821,7 @@ export function registerHostedHarnessSessionRoutes(
|
||||
() => error(res, 503, 'hosted_mcp_status_failed'),
|
||||
)
|
||||
.finally(() => {
|
||||
session.mcpRecovering = false;
|
||||
session.mcpRecovering -= 1;
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -366,8 +366,9 @@ export async function stopParkedRuntimeExecutions(input: {
|
||||
* original `executionCallId` — the Broker's durable record keeps that
|
||||
* exactly-once — commits the tool results and lets the checkpoint reach
|
||||
* `results_ready` before the caller answers. A passive load adopts the
|
||||
* dead owner's Runtime Session and reads execution states for the
|
||||
* cancellation path; it never dispatches.
|
||||
* original Runtime Session and reads execution states for the cancellation
|
||||
* path; it never dispatches. It reports that adoption before fallible reads,
|
||||
* so the caller retains the lease even when no recovery report returns.
|
||||
*
|
||||
* A parked state that can be taken over answers `recovered`; one that is
|
||||
* deterministically unrecoverable answers `declined` with a typed reason.
|
||||
@@ -389,6 +390,7 @@ export async function recoverHostedRuntimeTurn(input: {
|
||||
promptId: string;
|
||||
brokerOptions: HostedWorkspaceBrokerOptions;
|
||||
passive: boolean;
|
||||
onPassiveRuntimeAcquired?: (runtimeSessionId: string) => void;
|
||||
/** Set when the calling Session already holds the Runtime lease from an
|
||||
* earlier recovery of the same parked Turn (a re-answered load whose
|
||||
* first reply was lost): a failed re-acquire then says nothing about
|
||||
@@ -548,7 +550,7 @@ export async function recoverHostedRuntimeTurn(input: {
|
||||
// The passive path never compensation-releases: a release persists the
|
||||
// record as RELEASED, every retried acquire of the same identity then
|
||||
// conflicts with 409 runtime_session_not_acquirable, and a load that
|
||||
// throws leaves no harness-side record a route could hand back. The
|
||||
// throws before registration leaves no owner a route could hand back. The
|
||||
// adoption instead stays owed to the retried takeover, which re-acquires
|
||||
// a READY session under the same identity idempotently server-side; a
|
||||
// load that reports successfully hands the lease to the cancel route.
|
||||
@@ -557,6 +559,7 @@ export async function recoverHostedRuntimeTurn(input: {
|
||||
// follow-up, not settled by this change.
|
||||
await broker.acquire();
|
||||
acquiredRuntime = true;
|
||||
input.onPassiveRuntimeAcquired?.(broker.runtimeSessionId);
|
||||
}
|
||||
if (pending.length > 0) {
|
||||
if (passive) {
|
||||
|
||||
@@ -236,7 +236,15 @@ unarchive restores it to closed. Rename waits for the Harness to durably commit
|
||||
while retaining its receipt and request digest. The same
|
||||
key retries the same content with the replay flag set; changed content or a
|
||||
different Session conflicts. A successful concurrent request can still complete
|
||||
the receipt, and a failing sibling cannot overwrite that completed outcome.
|
||||
the receipt, and a failing sibling cannot overwrite that completed outcome. It
|
||||
cannot complete a retired receipt once a later rename has completed either: that
|
||||
sibling answers `409 session_mutation_superseded` and the newer public SQL title stays.
|
||||
The Harness title was already written before this check; ordering overlapping
|
||||
Harness writes remains a follow-up tracked in #13269. A
|
||||
same-key request sent after the later rename is the newest request and still
|
||||
applies, including when a concurrent sibling retires its receipt again. Each
|
||||
new attempt records the Session's current journal sequence on its command row;
|
||||
existing receipts use their original requested event until they are retried.
|
||||
Retries do not re-append the original `requested` event. If the command store
|
||||
is unavailable during cleanup, the original API failure is preserved and the
|
||||
same key can resume its receipt when storage returns. Only an in-flight
|
||||
@@ -493,8 +501,10 @@ not a filesystem sandbox.
|
||||
Later Turns may be submitted by the Session's creator under the
|
||||
same opt-in while they can still read and create in the Workspace (the
|
||||
per-caller `workspaceTurns` capability flag reflects the caller's current
|
||||
grants and the Workspace registry's `ACTIVE` state), and the creator may cancel
|
||||
the Session's running Turns and rename the Session. Workspace close follows
|
||||
grants, the Workspace registry's `ACTIVE` state and the Workspace generation and storage the
|
||||
Session was bound to), and the creator may rename the Session. The creator may
|
||||
also cancel a running Turn while they can still read the Workspace, under the
|
||||
cancel rule below. Workspace close follows
|
||||
its separate close capability and lifecycle admission. Archive, delete and
|
||||
unarchive follow their separate retention capabilities after reliable Workspace
|
||||
close. Controlled cwd changes ship below (W2); broad Workspace capability
|
||||
@@ -745,9 +755,25 @@ does not provide physical isolation or recovery after worker-only death.
|
||||
Public bound Turn admission is limited to the opt-in initial file Turn described
|
||||
in G0 above and to later Turns submitted by the Session's creator under the same
|
||||
opt-in while they can still read and create in the Workspace (the per-caller
|
||||
`workspaceTurns` capability flag reflects the caller's current grants and the
|
||||
registry's `ACTIVE` state); the creator may also cancel the Session's running
|
||||
Turns and rename the Session. Later Turns run
|
||||
`workspaceTurns` capability flag reflects the caller's current grants, the
|
||||
registry's `ACTIVE` state and the Workspace generation and storage the Session was bound
|
||||
to); the creator may also rename the Session. Cancelling aborts work that is
|
||||
already running, so the creator may cancel a running Turn while they can still
|
||||
read the Workspace and the deployment still enables Workspace files, even after
|
||||
their create grant is revoked, the Workspace
|
||||
starts draining or it is re-registered. A live cancel reuses the owner's
|
||||
resident Harness attachment. A cold connector cache passively re-attaches for
|
||||
the persisted cancellation after checking the frozen Session binding and exact
|
||||
identity, without depending on current creation grants, registry state or mount
|
||||
readiness. A resident passive load returns the original connection after scope
|
||||
and profile checks; a lost passive recovery reply can be retried without driving
|
||||
work. Passive recovery may adopt the original Runtime and query status without
|
||||
preparing or executing work; on the cancellation path, its lease stays owed
|
||||
through lost replies and retryable refusals until terminal success or teardown.
|
||||
New API cancellation requests still require read access, while already
|
||||
accepted cancellations continue if it is subsequently revoked. New work always
|
||||
rechecks execution authority. Broker/worker process death and an original prompt
|
||||
admission with a lost reply retain their separate recovery limitations. Later Turns run
|
||||
under the creator's Workspace grants, so any other actor keeps the existing
|
||||
refusal: `workspace_unavailable` when the actor can read the Workspace,
|
||||
`session_not_found` when they cannot. Public close follows its separate close
|
||||
|
||||
+2
-2
@@ -84,8 +84,8 @@ public final class ApiModels {
|
||||
}
|
||||
|
||||
public record WebShellSessionCapabilities(boolean tasks, boolean artifacts, boolean actions,
|
||||
boolean workspaceTurns, boolean sessionClose, boolean sessionArchive, boolean sessionUnarchive,
|
||||
boolean sessionDelete) {
|
||||
boolean workspaceTurns, boolean sessionClose, boolean sessionArchive,
|
||||
boolean sessionUnarchive, boolean sessionDelete) {
|
||||
}
|
||||
|
||||
@JsonInclude(JsonInclude.Include.NON_NULL)
|
||||
|
||||
+39
-14
@@ -209,7 +209,7 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
boolean loadExisting, boolean passiveManagedRuntimeRecovery) {
|
||||
try {
|
||||
return doCreateOrLoad(tenantId, sessionId, loadExisting,
|
||||
passiveManagedRuntimeRecovery);
|
||||
passiveManagedRuntimeRecovery, false);
|
||||
} catch (HostedHarnessGenerationException error) {
|
||||
adoptGeneration(error);
|
||||
throw error;
|
||||
@@ -217,7 +217,8 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
}
|
||||
|
||||
private Attachment doCreateOrLoad(String tenantId, String sessionId,
|
||||
boolean loadExisting, boolean passiveManagedRuntimeRecovery) {
|
||||
boolean loadExisting, boolean passiveManagedRuntimeRecovery,
|
||||
boolean actionResponse) {
|
||||
SessionRecord session = sessions.requireSession(tenantId, sessionId);
|
||||
if (session.workspace() != null) {
|
||||
if (!isWorkspaceFilesAvailable()) {
|
||||
@@ -227,7 +228,12 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
throw new IllegalStateException("Hosted Workspace Sessions"
|
||||
+ " require the Managed Action store");
|
||||
}
|
||||
if (passiveManagedRuntimeRecovery) {
|
||||
if (actionResponse) {
|
||||
workspaceExecution.authorizeActionResponse(session);
|
||||
if (!passiveManagedRuntimeRecovery) {
|
||||
workspaceExecution.verifyMountForProbe(session.workspace());
|
||||
}
|
||||
} else if (passiveManagedRuntimeRecovery) {
|
||||
workspaceExecution.authorizePassiveAttachment(session);
|
||||
} else {
|
||||
workspaceExecution.authorize(session);
|
||||
@@ -299,7 +305,7 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
private Admission doSubmit(String tenantId, String sessionId,
|
||||
String promptId,
|
||||
List<Map<String, Object>> input, String payloadDigest) {
|
||||
requireReadyForNewWork(tenantId, sessionId);
|
||||
requireReadyForNewWork(tenantId, sessionId, false);
|
||||
SubmitHarnessTurn.Builder builder = SubmitHarnessTurn.builder()
|
||||
.session(attachment(tenantId, sessionId, true))
|
||||
.promptId(promptId)
|
||||
@@ -327,7 +333,7 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
private Admission doContinueManagedRuntime(String tenantId,
|
||||
String sessionId, String promptId, String checkpointId,
|
||||
String activationId) {
|
||||
requireReadyForNewWork(tenantId, sessionId);
|
||||
requireReadyForNewWork(tenantId, sessionId, false);
|
||||
// Resolve the attachment BEFORE fetching the client: the resolution
|
||||
// may block on a create/load round trip, and an adoption closing the
|
||||
// captured client during that window would strand this call on a
|
||||
@@ -355,7 +361,7 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
private Admission doCancelManagedRuntime(String tenantId,
|
||||
String sessionId, String promptId, String checkpointId,
|
||||
String activationId) {
|
||||
HarnessSessionRef ref = attachment(tenantId, sessionId, false);
|
||||
HarnessSessionRef ref = cancellationAttachment(tenantId, sessionId);
|
||||
PromptReceipt receipt = client().cancelManagedRuntime(
|
||||
new CancelManagedRuntime(ref,
|
||||
promptId, checkpointId, activationId));
|
||||
@@ -425,8 +431,14 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
String sessionId,
|
||||
String actionId,
|
||||
JsonNode response) {
|
||||
requireReadyForNewWork(tenantId, sessionId);
|
||||
HarnessSessionRef ref = attachment(tenantId, sessionId, true);
|
||||
requireReadyForNewWork(tenantId, sessionId, true);
|
||||
AttachmentKey key = new AttachmentKey(tenantId, sessionId);
|
||||
HarnessSessionRef ref = attachments.get(key);
|
||||
if (ref == null) {
|
||||
doCreateOrLoad(tenantId, sessionId, true,
|
||||
workspaceExecution.verifiedRecoveryEnabled(), true);
|
||||
ref = attachments.get(key);
|
||||
}
|
||||
client().resolveAction(
|
||||
ref,
|
||||
actionId,
|
||||
@@ -446,8 +458,9 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
}
|
||||
|
||||
private void doCancel(String tenantId, String sessionId) {
|
||||
HarnessSessionRef ref = attachment(tenantId, sessionId, false);
|
||||
HarnessSessionRef ref = cancellationAttachment(tenantId, sessionId);
|
||||
client().cancelTurn(ref);
|
||||
pendingRecovery.remove(new AttachmentKey(tenantId, sessionId));
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -505,16 +518,28 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
return attachment;
|
||||
}
|
||||
|
||||
private void requireReadyForNewWork(String tenantId, String sessionId) {
|
||||
if (!workspaceExecution.verifiedRecoveryEnabled()) {
|
||||
return;
|
||||
private HarnessSessionRef cancellationAttachment(String tenantId, String sessionId) {
|
||||
AttachmentKey key = new AttachmentKey(tenantId, sessionId);
|
||||
HarnessSessionRef attached = attachments.get(key);
|
||||
if (attached == null) {
|
||||
recoverManagedRuntime(tenantId, sessionId, true);
|
||||
attached = attachments.get(key);
|
||||
}
|
||||
return attached;
|
||||
}
|
||||
|
||||
private void requireReadyForNewWork(String tenantId, String sessionId, boolean actionResponse) {
|
||||
SessionRecord session = sessions.requireSession(tenantId, sessionId);
|
||||
if (session.workspace() != null) {
|
||||
if (!isWorkspaceFilesAvailable()) {
|
||||
throw new IllegalStateException("Hosted Workspace files are disabled");
|
||||
}
|
||||
workspaceExecution.authorize(session);
|
||||
if (actionResponse) {
|
||||
workspaceExecution.authorizeActionResponse(session);
|
||||
workspaceExecution.verifyMountForProbe(session.workspace());
|
||||
} else {
|
||||
workspaceExecution.authorize(session);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -648,7 +673,7 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
||||
+ " require the Managed Action store");
|
||||
}
|
||||
if (cancellation) {
|
||||
workspaceExecution.authorizePassiveAttachment(session);
|
||||
workspaceExecution.authorizeCancellation(session);
|
||||
} else {
|
||||
workspaceExecution.authorize(session);
|
||||
}
|
||||
|
||||
+7
@@ -9,6 +9,7 @@ import com.alibaba.qwen.code.managedagent.store.ManagedActionStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedActionStore.Action;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedActionStore.Response;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.OperationRecord;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
@@ -115,6 +116,12 @@ public class ActionResponseCoordinator {
|
||||
if (settled(op, actions.response(tenant, session, operation))) {
|
||||
return;
|
||||
}
|
||||
if (error instanceof RuntimeBrokerException failure
|
||||
&& !failure.isRetryable()
|
||||
&& "workspace_unavailable".equals(failure.getCode())) {
|
||||
actions.complete(op, owner, failure.getCode(), null, clock.millis());
|
||||
return;
|
||||
}
|
||||
long delay =
|
||||
HarnessCoordinator.retryDelay(
|
||||
dispatch.getRetryInitialDelay(),
|
||||
|
||||
+20
-11
@@ -144,7 +144,7 @@ public class HarnessCoordinator {
|
||||
public void cancel(String tenantId, String sessionId, String turnId) {
|
||||
dispatch(tenantId, sessionId, turnId);
|
||||
executor.execute(() -> cancelAdmittedTurn(tenantId, sessionId,
|
||||
turnId));
|
||||
turnId, false));
|
||||
}
|
||||
|
||||
@Scheduled(fixedDelayString =
|
||||
@@ -180,7 +180,7 @@ public class HarnessCoordinator {
|
||||
leaseLost.set(true);
|
||||
} else if (!leaseLost.get()) {
|
||||
executor.execute(() -> cancelAdmittedTurn(
|
||||
tenantId, sessionId, turnId));
|
||||
tenantId, sessionId, turnId, true));
|
||||
}
|
||||
} catch (RuntimeException error) {
|
||||
leaseLost.set(true);
|
||||
@@ -857,7 +857,7 @@ public class HarnessCoordinator {
|
||||
}
|
||||
|
||||
private void cancelAdmittedTurn(String tenantId, String sessionId,
|
||||
String turnId) {
|
||||
String turnId, boolean renewalDriven) {
|
||||
try {
|
||||
TurnRecord turn = store.findTurn(tenantId, sessionId, turnId)
|
||||
.orElse(null);
|
||||
@@ -875,17 +875,22 @@ public class HarnessCoordinator {
|
||||
&& !harness.isWorkspaceFilesAvailable()) {
|
||||
return;
|
||||
}
|
||||
// Reuse the admitted attachment: attaching would recheck grants
|
||||
// needed for new work and could replace the running attachment.
|
||||
// Reuse the admitted owner; a cold connector cache re-attaches
|
||||
// only for the persisted cancellation, without requiring the
|
||||
// authority that admits new work.
|
||||
if (session.harnessBootId() != null && store.bindHarness(tenantId,
|
||||
sessionId, turnId, owner, session.harnessBootId())) {
|
||||
// Renewal requeues this poller while the Turn is still
|
||||
// settling, so only one concurrent poller may send the
|
||||
// cancel; a failed send releases the claim so the next
|
||||
// renewal retries it.
|
||||
// settling, so only one renewal-driven poller may send the
|
||||
// cancel once per event epoch; a failed send releases the
|
||||
// claim so the next renewal retries it. A public cancel()
|
||||
// call always pays its own send: a takeover load re-armed
|
||||
// after a transient failure must be settled by the send it
|
||||
// re-requests, and the plain cancel is idempotent at the
|
||||
// daemon under the same identities.
|
||||
String cancelKey = key(tenantId, sessionId, turnId) + "\n"
|
||||
+ turn.harnessEventEpoch();
|
||||
if (cancellations.add(cancelKey)) {
|
||||
if (!renewalDriven || cancellations.add(cancelKey)) {
|
||||
try {
|
||||
harness.cancel(session.tenantId(),
|
||||
session.sessionId());
|
||||
@@ -898,7 +903,9 @@ public class HarnessCoordinator {
|
||||
// The coded refusal is a failed plain send: release
|
||||
// the claim here and let the paced takeover below,
|
||||
// not the claim, gate re-sends of this path.
|
||||
cancellations.remove(cancelKey);
|
||||
if (renewalDriven) {
|
||||
cancellations.remove(cancelKey);
|
||||
}
|
||||
// The plain cancel route aborts only a live, in-memory
|
||||
// Turn: a parked Turn its dead generation owned answers
|
||||
// the coded refusal, and re-issuing that same cancel
|
||||
@@ -939,7 +946,9 @@ public class HarnessCoordinator {
|
||||
recovery.getActivationId());
|
||||
}
|
||||
} catch (RuntimeException error) {
|
||||
cancellations.remove(cancelKey);
|
||||
if (renewalDriven) {
|
||||
cancellations.remove(cancelKey);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
+89
-12
@@ -31,6 +31,7 @@ import com.alibaba.qwen.code.managedagent.store.AgentStateStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedWorkspaceRegistry;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.Admission;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.CommandRecord;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.EventRecord;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.EventPage;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.ItemPartRecord;
|
||||
@@ -51,6 +52,7 @@ import java.util.Base64;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
import java.util.function.BooleanSupplier;
|
||||
import java.util.regex.Pattern;
|
||||
@@ -223,17 +225,25 @@ public class ManagedAgentService {
|
||||
String idempotencyKey, String sessionId,
|
||||
List<InputBlock> blocks) {
|
||||
validateIdempotencyKey(idempotencyKey);
|
||||
requireSubmitter(tenantId, actorId, sessionId);
|
||||
requireHarness();
|
||||
requireReadableSession(tenantId, actorId, sessionId);
|
||||
requireBoundCreator(tenantId, actorId, sessionId);
|
||||
List<Map<String, Object>> input = input(blocks, true);
|
||||
String requestDigest = digests.digest(Map.of(
|
||||
"sessionId", sessionId, "input", input));
|
||||
// Replay is a read of the recorded admission, not a re-admission:
|
||||
// answer a same-key retry before the admission gate, which may now
|
||||
// refuse on state that postdates the recorded admission (a
|
||||
// re-registered Workspace, a revoked grant, a DRAINING registry),
|
||||
// or the client can never recover the Turn it was given.
|
||||
Admission replay = replay(tenantId, SUBMIT, idempotencyKey,
|
||||
requestDigest);
|
||||
if (replay != null) {
|
||||
requireHarness();
|
||||
dispatch(tenantId, replay);
|
||||
return response(replay);
|
||||
}
|
||||
requireSubmitter(tenantId, actorId, sessionId);
|
||||
requireHarness();
|
||||
String payloadDigest = SubmitHarnessTurn.computePayloadDigest(input);
|
||||
Admission admission;
|
||||
try {
|
||||
@@ -251,7 +261,7 @@ public class ManagedAgentService {
|
||||
public CommandAdmission cancelTurn(String tenantId, String actorId,
|
||||
String idempotencyKey, String sessionId, String turnId) {
|
||||
validateIdempotencyKey(idempotencyKey);
|
||||
requireSubmitter(tenantId, actorId, sessionId);
|
||||
requireCanceller(tenantId, actorId, sessionId);
|
||||
String requestDigest = digests.digest(Map.of(
|
||||
"sessionId", sessionId, "turnId", turnId));
|
||||
Admission replay = replay(tenantId, CANCEL, idempotencyKey,
|
||||
@@ -281,10 +291,32 @@ public class ManagedAgentService {
|
||||
String tenantId, String actorId, String idempotencyKey, String sessionId,
|
||||
String title) {
|
||||
validateIdempotencyKey(idempotencyKey);
|
||||
requireSubmitter(tenantId, actorId, sessionId);
|
||||
requireReadableSession(tenantId, actorId, sessionId);
|
||||
requireBoundCreator(tenantId, actorId, sessionId);
|
||||
String effectiveTitle = validRenameTitle(title);
|
||||
String requestDigest = digests.digest(Map.of(
|
||||
"sessionId", sessionId, "title", effectiveTitle));
|
||||
// A completed rename is answered from its record before the
|
||||
// admission gate: the gate may now refuse on state that postdates
|
||||
// the recorded outcome, and a same-key retry must not lose it.
|
||||
// A PENDING row falls through so beginSessionMutation answers it as
|
||||
// replayed and the retry re-drives the unfinished mutation.
|
||||
Optional<CommandRecord> recorded = store.findCommand(tenantId,
|
||||
RENAME, idempotencyKey);
|
||||
if (recorded.isPresent()) {
|
||||
CommandRecord existing = recorded.get();
|
||||
if (!existing.requestDigest().equals(requestDigest)
|
||||
|| !existing.sessionId().equals(sessionId)) {
|
||||
throw new ApiException(HttpStatus.CONFLICT,
|
||||
"idempotency_conflict",
|
||||
"The idempotency key was reused with different content.");
|
||||
}
|
||||
if ("COMPLETED".equals(existing.status())) {
|
||||
return new SessionMutationResult<>(getPublicSession(tenantId,
|
||||
sessionId), true);
|
||||
}
|
||||
}
|
||||
requireSubmitter(tenantId, actorId, sessionId);
|
||||
SessionMutationCommand command = store.beginSessionMutation(tenantId,
|
||||
RENAME, idempotencyKey, requestDigest, sessionId,
|
||||
SessionMutationKind.RENAME);
|
||||
@@ -638,7 +670,7 @@ public class ManagedAgentService {
|
||||
session.sessionId()))
|
||||
.map(session -> session.workspace().getWorkspaceId())
|
||||
.collect(java.util.stream.Collectors.toSet());
|
||||
Map<String, ManagedWorkspaceRegistry.WorkspaceSummary> grants =
|
||||
Map<String, ManagedWorkspaceRegistry.ReadableGrant> grants =
|
||||
grantWorkspaces.isEmpty() ? Map.of()
|
||||
: workspaces.findReadable(tenantId, actorId,
|
||||
grantWorkspaces);
|
||||
@@ -820,8 +852,9 @@ public class ManagedAgentService {
|
||||
|
||||
// Later Turns of a Workspace-bound Session run under the creator's
|
||||
// Workspace grants (WorkspaceExecutionStore.authorize), so only the
|
||||
// creator may submit or cancel them or rename the Session, and only with
|
||||
// Workspace files enabled. Everyone else keeps the existing refusal.
|
||||
// creator may submit them or rename the Session, and only with Workspace
|
||||
// files enabled. Everyone else keeps the existing refusal. Cancelling
|
||||
// has its own, narrower rule (requireCanceller).
|
||||
private void requireSubmitter(String tenantId, String actorId,
|
||||
String sessionId) {
|
||||
SessionRecord session = store.requireSession(tenantId, sessionId);
|
||||
@@ -830,6 +863,36 @@ public class ManagedAgentService {
|
||||
}
|
||||
}
|
||||
|
||||
// Replay skips the admission gate, so a bound Session's recorded
|
||||
// admission answers only its creator. The creation receipt survives a
|
||||
// revoked grant, a draining Workspace and a re-registration, so the
|
||||
// creator's own same-key retry still replays.
|
||||
private void requireBoundCreator(String tenantId, String actorId,
|
||||
String sessionId) {
|
||||
SessionRecord session = store.requireSession(tenantId, sessionId);
|
||||
if (session.workspace() != null
|
||||
&& !workspaces.createdSession(tenantId, actorId, sessionId)) {
|
||||
requireLegacyWorkspace(session, actorId);
|
||||
}
|
||||
}
|
||||
|
||||
// Cancelling aborts work that is already running, so it needs only what
|
||||
// identifies the creator, not the grants that admit new work: the
|
||||
// creator who can still read the Workspace may cancel while can_create is
|
||||
// revoked, the Workspace is draining or it was re-registered. The shape
|
||||
// term stays: a Session that can no longer execute keeps the refusal.
|
||||
private void requireCanceller(String tenantId, String actorId,
|
||||
String sessionId) {
|
||||
SessionRecord session = store.requireSession(tenantId, sessionId);
|
||||
if (!maySubmitShape(session)
|
||||
|| !workspaces.canRead(session.tenantId(), actorId,
|
||||
session.workspace().getWorkspaceId())
|
||||
|| !workspaces.createdSession(session.tenantId(), actorId,
|
||||
session.sessionId())) {
|
||||
requireLegacyWorkspace(session, actorId);
|
||||
}
|
||||
}
|
||||
|
||||
private boolean maySubmitWorkspaceTurn(SessionRecord session,
|
||||
String actorId) {
|
||||
if (!maySubmitShape(session)) {
|
||||
@@ -848,20 +911,34 @@ public class ManagedAgentService {
|
||||
ManagedWorkspaceRegistry.WorkspaceSummary summary =
|
||||
workspaces.findReadable(session.tenantId(), actorId,
|
||||
session.workspace().getWorkspaceId());
|
||||
return summary != null && summary.canCreateSession();
|
||||
// Execution also requires the Workspace generation and storage the
|
||||
// Session was bound to; after a re-registration it refuses, so
|
||||
// admission must refuse first instead of accepting a Turn that fails.
|
||||
return summary != null && summary.canCreateSession()
|
||||
&& workspaces.bindingCurrent(session.tenantId(),
|
||||
session.workspace().getWorkspaceId(),
|
||||
session.workspace().getWorkspaceGeneration(),
|
||||
session.workspace().getStorageId());
|
||||
}
|
||||
|
||||
// The page twin of the singular: the same rule answered from the batch
|
||||
// reads the assembler already made.
|
||||
// reads the assembler already made. The grant batch carries the registry's
|
||||
// binding stamp, so a re-registration drops the capability here exactly as
|
||||
// bindingCurrent drops it in the singular.
|
||||
private boolean maySubmitWorkspaceTurn(SessionRecord session,
|
||||
Set<String> creatorOwns,
|
||||
Map<String, ManagedWorkspaceRegistry.WorkspaceSummary> grants) {
|
||||
Map<String, ManagedWorkspaceRegistry.ReadableGrant> grants) {
|
||||
if (!maySubmitShape(session)
|
||||
|| !creatorOwns.contains(session.sessionId())) {
|
||||
return false;
|
||||
}
|
||||
var summary = grants.get(session.workspace().getWorkspaceId());
|
||||
return summary != null && summary.canCreateSession();
|
||||
var grant = grants.get(session.workspace().getWorkspaceId());
|
||||
if (grant == null || !grant.canCreateSession()) {
|
||||
return false;
|
||||
}
|
||||
var binding = session.workspace();
|
||||
return grant.workspaceGeneration() == binding.getWorkspaceGeneration()
|
||||
&& grant.storageId().equals(binding.getStorageId());
|
||||
}
|
||||
|
||||
private boolean maySubmitShape(SessionRecord session) {
|
||||
|
||||
+48
-3
@@ -577,9 +577,11 @@ public class ManagedAgentStore implements AgentStateStore {
|
||||
requireNoOpenOperation(tenantId, sessionId);
|
||||
validateMutationStatus(session, kind);
|
||||
jdbc.update("UPDATE managed_agent_command SET command_status ="
|
||||
+ " 'PENDING', updated_at = ? WHERE tenant_id = ?"
|
||||
+ " 'PENDING', mutation_attempt_sequence = ?,"
|
||||
+ " updated_at = ? WHERE tenant_id = ?"
|
||||
+ " AND operation = ? AND idempotency_key = ?",
|
||||
clock.millis(), tenantId, operation, idempotencyKey);
|
||||
session.lastSequence(), clock.millis(), tenantId,
|
||||
operation, idempotencyKey);
|
||||
return new SessionMutationCommand(sessionId, "PENDING", true);
|
||||
}
|
||||
return new SessionMutationCommand(sessionId, command.status(),
|
||||
@@ -593,6 +595,10 @@ public class ManagedAgentStore implements AgentStateStore {
|
||||
long now = clock.millis();
|
||||
insertCommand(tenantId, operation, idempotencyKey, requestDigest,
|
||||
sessionId, null, "PENDING", session.status(), now);
|
||||
jdbc.update("UPDATE managed_agent_command SET mutation_attempt_sequence"
|
||||
+ " = ? WHERE tenant_id = ? AND operation = ?"
|
||||
+ " AND idempotency_key = ?",
|
||||
session.lastSequence(), tenantId, operation, idempotencyKey);
|
||||
// Older retired commands may have left their requested event behind.
|
||||
String requestedSource = mutationSource(operation, idempotencyKey,
|
||||
"requested");
|
||||
@@ -631,6 +637,17 @@ public class ManagedAgentStore implements AgentStateStore {
|
||||
throw new IllegalStateException(
|
||||
"Session mutation command has an unknown status");
|
||||
}
|
||||
// A retired receipt still completes for a sibling that entered the
|
||||
// Harness before the retirement, but never over a later mutation:
|
||||
// while it was FAILED another key could begin and complete, and
|
||||
// completing this one now would revert that newer outcome.
|
||||
if ("FAILED".equals(command.status()) && supersededByLaterMutation(
|
||||
tenantId, sessionId, operation, idempotencyKey, kind)) {
|
||||
throw new ApiException(HttpStatus.CONFLICT,
|
||||
"session_mutation_superseded",
|
||||
"A later change to the Session completed after this"
|
||||
+ " request was retired.");
|
||||
}
|
||||
if (session.workspace() != null) {
|
||||
WorkspaceMigrationAdmission.requireOpen(jdbc, tenantId, session.workspace().getStorageId());
|
||||
}
|
||||
@@ -745,7 +762,7 @@ public class ManagedAgentStore implements AgentStateStore {
|
||||
if (!workspaces.canRead(session.tenantId(), actorId, session.workspace().getWorkspaceId())) {
|
||||
throw new ApiException(HttpStatus.NOT_FOUND, "session_not_found", "The Session was not found.");
|
||||
}
|
||||
if (!workspaces.isSessionCreator(session.tenantId(), session.sessionId(), actorId)) {
|
||||
if (!workspaces.createdSession(session.tenantId(), actorId, session.sessionId())) {
|
||||
throw new ApiException(HttpStatus.FORBIDDEN, "session_operation_forbidden",
|
||||
"Only the Session creator may manage it.");
|
||||
}
|
||||
@@ -2913,6 +2930,34 @@ public class ManagedAgentStore implements AgentStateStore {
|
||||
+ "." + phase;
|
||||
}
|
||||
|
||||
// The Session lock orders each new attempt against committed events.
|
||||
// Legacy receipts fall back to their original requested event; a
|
||||
// re-drive refreshes the boundary without appending another event.
|
||||
private boolean supersededByLaterMutation(String tenantId,
|
||||
String sessionId, String operation, String idempotencyKey,
|
||||
SessionMutationKind kind) {
|
||||
List<Long> attempts = jdbc.queryForList("SELECT COALESCE("
|
||||
+ " c.mutation_attempt_sequence, e.sequence_id) FROM"
|
||||
+ " managed_agent_command c LEFT JOIN managed_agent_event e"
|
||||
+ " ON e.tenant_id = c.tenant_id AND"
|
||||
+ " e.session_id = c.session_id AND e.source_key = ?"
|
||||
+ " WHERE c.tenant_id = ? AND c.session_id = ? AND"
|
||||
+ " c.operation = ? AND c.idempotency_key = ?",
|
||||
Long.class, mutationSource(operation, idempotencyKey,
|
||||
"requested"), tenantId, sessionId, operation,
|
||||
idempotencyKey);
|
||||
if (attempts.isEmpty() || attempts.getFirst() == null) {
|
||||
return false;
|
||||
}
|
||||
Integer later = jdbc.queryForObject("SELECT COUNT(*) FROM"
|
||||
+ " managed_agent_event WHERE tenant_id = ? AND"
|
||||
+ " session_id = ? AND sequence_id > ? AND"
|
||||
+ " event_type = ? AND source_key LIKE 'control:%'",
|
||||
Integer.class, tenantId, sessionId, attempts.getFirst(),
|
||||
mutationEvent(kind, "completed"));
|
||||
return later != null && later > 0;
|
||||
}
|
||||
|
||||
private static String mutationSource(String operation,
|
||||
String idempotencyKey, String phase) {
|
||||
return "control:" + operation + ":" + idempotencyKey + ":" + phase;
|
||||
|
||||
+46
-12
@@ -49,10 +49,6 @@ public class ManagedWorkspaceRegistry {
|
||||
Integer.class, tenantId, sessionId, tenantId, key).isEmpty();
|
||||
}
|
||||
|
||||
public boolean isSessionCreator(String tenantId, String sessionId, String actorId) {
|
||||
return createdSession(tenantId, actorId, sessionId);
|
||||
}
|
||||
|
||||
/** The batch twin of createdSession for a page of Session ids. */
|
||||
public java.util.Set<String> createdSessions(String tenantId,
|
||||
String actorId, List<String> sessionIds) {
|
||||
@@ -154,8 +150,13 @@ public class ManagedWorkspaceRegistry {
|
||||
return rows.isEmpty() ? null : rows.getFirst();
|
||||
}
|
||||
|
||||
/** The batch twin of findReadable for a page of Workspace ids. */
|
||||
public java.util.Map<String, WorkspaceSummary> findReadable(
|
||||
/**
|
||||
* The batch twin of findReadable for a page of Workspace ids. It carries
|
||||
* the registry's current binding stamp as well, so a page answers the
|
||||
* creator-submit gate from this one read instead of a per-row
|
||||
* bindingCurrent.
|
||||
*/
|
||||
public java.util.Map<String, ReadableGrant> findReadable(
|
||||
String tenantId, String actorId,
|
||||
java.util.Collection<String> workspaceIds) {
|
||||
if (workspaceIds.isEmpty()) {
|
||||
@@ -173,9 +174,10 @@ public class ManagedWorkspaceRegistry {
|
||||
arguments.addAll(workspaceIds);
|
||||
arguments.add(tenantId);
|
||||
arguments.add(key);
|
||||
List<WorkspaceSummary> rows = jdbc.query(
|
||||
"SELECT r.workspace_id, r.display_name, r.state,"
|
||||
+ " a.can_create FROM managed_workspace_registry r"
|
||||
List<ReadableGrant> rows = jdbc.query(
|
||||
"SELECT r.workspace_id, r.workspace_generation,"
|
||||
+ " r.storage_id, r.state, a.can_create"
|
||||
+ " FROM managed_workspace_registry r"
|
||||
+ " JOIN managed_workspace_access a ON"
|
||||
+ " a.tenant_id = r.tenant_id"
|
||||
+ " AND a.workspace_id = r.workspace_id"
|
||||
@@ -190,10 +192,10 @@ public class ManagedWorkspaceRegistry {
|
||||
+ " AND CAST(CONCAT(r.tenant_id, '!') AS BINARY(513))"
|
||||
+ " = CAST(CONCAT(?, '!') AS BINARY(513))"
|
||||
+ " AND a.actor_id = ? AND a.can_read = TRUE",
|
||||
(result, row) -> summary(result), arguments.toArray());
|
||||
java.util.Map<String, WorkspaceSummary> result =
|
||||
(result, row) -> readableGrant(result), arguments.toArray());
|
||||
java.util.Map<String, ReadableGrant> result =
|
||||
new java.util.HashMap<>(rows.size() * 2);
|
||||
for (WorkspaceSummary row : rows) {
|
||||
for (ReadableGrant row : rows) {
|
||||
result.put(row.workspaceId(), row);
|
||||
}
|
||||
return result;
|
||||
@@ -224,10 +226,42 @@ public class ManagedWorkspaceRegistry {
|
||||
&& "ACTIVE".equals(state));
|
||||
}
|
||||
|
||||
private static ReadableGrant readableGrant(ResultSet result)
|
||||
throws SQLException {
|
||||
String state = result.getString("state");
|
||||
return new ReadableGrant(result.getString("workspace_id"),
|
||||
result.getLong("workspace_generation"),
|
||||
result.getString("storage_id"),
|
||||
result.getBoolean("can_create")
|
||||
&& "ACTIVE".equals(state));
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the registry still holds the Workspace generation and storage a
|
||||
* Session was bound to; a re-registration changes them.
|
||||
*/
|
||||
public boolean bindingCurrent(String tenantId, String workspaceId,
|
||||
long generation, String storageId) {
|
||||
return !jdbc.queryForList("SELECT 1 FROM managed_workspace_registry"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ?"
|
||||
+ " AND workspace_generation = ? AND storage_id = ?",
|
||||
Integer.class, tenantId, workspaceId, generation, storageId)
|
||||
.isEmpty();
|
||||
}
|
||||
|
||||
public record WorkspaceSummary(String workspaceId, String displayName,
|
||||
String state, boolean canCreateSession) {
|
||||
}
|
||||
|
||||
/**
|
||||
* A readable Workspace with the generation and storage the registry holds
|
||||
* now, so a batch caller can tell whether a Session's recorded binding is
|
||||
* still the current one.
|
||||
*/
|
||||
public record ReadableGrant(String workspaceId, long workspaceGeneration,
|
||||
String storageId, boolean canCreateSession) {
|
||||
}
|
||||
|
||||
public ResolvedBinding resolveForCreation(String tenantId,
|
||||
String actorId, WorkspaceSelection selection) {
|
||||
if (!TransactionSynchronizationManager.isActualTransactionActive()) {
|
||||
|
||||
+94
-36
@@ -85,14 +85,32 @@ public class WorkspaceExecutionStore {
|
||||
}
|
||||
|
||||
public void authorizePassiveAttachment(SessionRecord session) {
|
||||
authorizePassiveAttachment(session, null);
|
||||
authorizeAttachment(session, false, false, null, false);
|
||||
}
|
||||
|
||||
private void authorizePassiveAttachment(SessionRecord session, com.alibaba.qwen.code.runtimebroker.RuntimeLifecycleAuthority authority) {
|
||||
authorizePassiveAttachment(session, authority, false);
|
||||
private void authorizePassiveAttachment(SessionRecord session,
|
||||
com.alibaba.qwen.code.runtimebroker.RuntimeLifecycleAuthority authority) {
|
||||
authorizeAttachment(session, false, false, authority, false);
|
||||
}
|
||||
|
||||
private void authorizePassiveAttachment(SessionRecord session, com.alibaba.qwen.code.runtimebroker.RuntimeLifecycleAuthority authority,
|
||||
private void authorizePassiveAttachment(SessionRecord session,
|
||||
com.alibaba.qwen.code.runtimebroker.RuntimeLifecycleAuthority authority, boolean legacyClose) {
|
||||
authorizeAttachment(session, false, false, authority, legacyClose);
|
||||
}
|
||||
|
||||
public void authorizeCancellation(SessionRecord session) {
|
||||
authorizeAttachment(session, true, false, null, false);
|
||||
}
|
||||
|
||||
// Action-response delivery only: a refusal stemming solely from
|
||||
// operator-mutable grants or registry state must not certify the
|
||||
// terminal verdict, so it answers with the retryable variant instead.
|
||||
public void authorizeActionResponse(SessionRecord session) {
|
||||
authorizeAttachment(session, false, true, null, false);
|
||||
}
|
||||
|
||||
private void authorizeAttachment(SessionRecord session, boolean cancellation,
|
||||
boolean actionResponse, com.alibaba.qwen.code.runtimebroker.RuntimeLifecycleAuthority authority,
|
||||
boolean legacyClose) {
|
||||
ContextBinding binding = session.workspace();
|
||||
String expectedStatus = authority == null && !legacyClose ? "ACTIVE" : session.status();
|
||||
@@ -105,7 +123,9 @@ public class WorkspaceExecutionStore {
|
||||
throw unavailable();
|
||||
}
|
||||
WorkspaceStorageKindGuard.requireLocalAlias(jdbc, binding.getTenantId(), binding.getStorageId());
|
||||
List<Boolean> grants = jdbc.query("SELECT s.tenant_id, s.session_id,"
|
||||
// Verdicts per matched row: 0 structural mismatch, 1 refused only by
|
||||
// operator-mutable grant or registry state, 2 authorized.
|
||||
List<Integer> grants = jdbc.query("SELECT s.tenant_id, s.session_id,"
|
||||
+ " s.agent_id AS session_agent, s.status AS session_status,"
|
||||
+ " s.deleted_at AS session_deleted_at,"
|
||||
+ " s.workspace_id AS session_workspace,"
|
||||
@@ -114,46 +134,74 @@ public class WorkspaceExecutionStore {
|
||||
+ " s.cwd_relative AS session_cwd,"
|
||||
+ " s.context_config_ref AS session_context,"
|
||||
+ " s.context_revision AS session_revision,"
|
||||
+ " s.workspace_config_ref, s.workspace_policy_ref,"
|
||||
+ " r.tenant_id AS registry_tenant, r.workspace_id,"
|
||||
+ " s.workspace_config_ref, s.workspace_policy_ref"
|
||||
+ (cancellation
|
||||
? ", t.tenant_id AS cancellation_tenant, t.session_id AS cancellation_session"
|
||||
: ", r.tenant_id AS registry_tenant, r.workspace_id,"
|
||||
+ " r.workspace_generation, r.storage_id, r.state,"
|
||||
+ " c.tenant_id AS command_tenant, c.session_id AS command_session,"
|
||||
+ " a.tenant_id AS access_tenant, a.workspace_id AS access_workspace,"
|
||||
+ " a.can_read, a.can_create FROM managed_agent_session s"
|
||||
+ " JOIN managed_workspace_registry r ON r.tenant_id = s.tenant_id"
|
||||
+ " a.can_read, a.can_create")
|
||||
+ " FROM managed_agent_session s"
|
||||
+ (cancellation
|
||||
? " JOIN managed_agent_turn t ON t.tenant_id = s.tenant_id"
|
||||
+ " AND t.session_id = s.session_id AND t.status = 'CANCELLING'"
|
||||
+ " AND (t.submission_attempted = TRUE OR t.harness_event_epoch IS NOT NULL)"
|
||||
: " JOIN managed_workspace_registry r ON r.tenant_id = s.tenant_id"
|
||||
+ " AND r.workspace_id = s.workspace_id"
|
||||
+ " JOIN managed_workspace_create_command c ON c.tenant_id = s.tenant_id"
|
||||
+ " AND c.session_id = s.session_id"
|
||||
+ " JOIN managed_workspace_access a ON a.tenant_id = r.tenant_id"
|
||||
+ " AND a.workspace_id = r.workspace_id AND a.actor_id = c.actor_id"
|
||||
+ " AND a.workspace_id = r.workspace_id AND a.actor_id = c.actor_id")
|
||||
+ " WHERE s.tenant_id = ? AND s.session_id = ?",
|
||||
(row, index) -> session.tenantId().equals(row.getString("tenant_id"))
|
||||
&& session.sessionId().equals(row.getString("session_id"))
|
||||
&& "qwen-code".equals(row.getString("session_agent"))
|
||||
&& expectedStatus.equals(row.getString("session_status"))
|
||||
&& row.getObject("session_deleted_at") == null
|
||||
&& binding.getWorkspaceId().equals(row.getString("session_workspace"))
|
||||
&& binding.getWorkspaceGeneration() == row.getLong("session_generation")
|
||||
&& binding.getStorageId().equals(row.getString("session_storage"))
|
||||
&& binding.getCwdRelative().equals(row.getString("session_cwd"))
|
||||
&& binding.getContextConfigRef().equals(row.getString("session_context"))
|
||||
&& binding.getContextRevision() == row.getLong("session_revision")
|
||||
&& session.tenantId().equals(row.getString("registry_tenant"))
|
||||
&& session.tenantId().equals(row.getString("command_tenant"))
|
||||
&& session.sessionId().equals(row.getString("command_session"))
|
||||
&& session.tenantId().equals(row.getString("access_tenant"))
|
||||
&& binding.getWorkspaceId().equals(row.getString("workspace_id"))
|
||||
&& binding.getWorkspaceId().equals(row.getString("access_workspace"))
|
||||
&& binding.getWorkspaceGeneration() == row.getLong("workspace_generation")
|
||||
&& binding.getStorageId().equals(row.getString("storage_id"))
|
||||
&& "ACTIVE".equals(row.getString("state"))
|
||||
&& row.getBoolean("can_read") && row.getBoolean("can_create")
|
||||
&& WorkspaceExecutionProfile.CONFIG_REF.equals(
|
||||
row.getString("workspace_config_ref"))
|
||||
&& WorkspaceExecutionProfile.POLICY_REF.equals(
|
||||
row.getString("workspace_policy_ref")),
|
||||
(row, index) -> {
|
||||
boolean structural = session.tenantId().equals(row.getString("tenant_id"))
|
||||
&& session.sessionId().equals(row.getString("session_id"))
|
||||
&& "qwen-code".equals(row.getString("session_agent"))
|
||||
&& expectedStatus.equals(row.getString("session_status"))
|
||||
&& row.getObject("session_deleted_at") == null
|
||||
&& binding.getWorkspaceId().equals(row.getString("session_workspace"))
|
||||
&& binding.getWorkspaceGeneration() == row.getLong("session_generation")
|
||||
&& binding.getStorageId().equals(row.getString("session_storage"))
|
||||
&& binding.getCwdRelative().equals(row.getString("session_cwd"))
|
||||
&& binding.getContextConfigRef().equals(row.getString("session_context"))
|
||||
&& binding.getContextRevision() == row.getLong("session_revision")
|
||||
&& WorkspaceExecutionProfile.CONFIG_REF.equals(
|
||||
row.getString("workspace_config_ref"))
|
||||
&& WorkspaceExecutionProfile.POLICY_REF.equals(
|
||||
row.getString("workspace_policy_ref"))
|
||||
// Cancellation was authorized when it was persisted;
|
||||
// retries must not depend on mutable creation grants.
|
||||
&& (cancellation
|
||||
? session.tenantId().equals(row.getString("cancellation_tenant"))
|
||||
&& session.sessionId().equals(row.getString("cancellation_session"))
|
||||
: session.tenantId().equals(row.getString("registry_tenant"))
|
||||
&& session.tenantId().equals(row.getString("command_tenant"))
|
||||
&& session.sessionId().equals(row.getString("command_session"))
|
||||
&& session.tenantId().equals(row.getString("access_tenant"))
|
||||
&& binding.getWorkspaceId().equals(row.getString("workspace_id"))
|
||||
&& binding.getWorkspaceId().equals(row.getString("access_workspace"))
|
||||
&& binding.getWorkspaceGeneration() == row.getLong("workspace_generation")
|
||||
&& binding.getStorageId().equals(row.getString("storage_id")));
|
||||
if (!structural) {
|
||||
return 0;
|
||||
}
|
||||
// Grants and registry state are operator-mutable: a
|
||||
// refusal stemming only from them is not the structural
|
||||
// verdict the terminal exit promises.
|
||||
return cancellation
|
||||
|| ("ACTIVE".equals(row.getString("state"))
|
||||
&& row.getBoolean("can_read") && row.getBoolean("can_create"))
|
||||
? 2 : 1;
|
||||
},
|
||||
session.tenantId(), session.sessionId());
|
||||
if (grants.size() != 1 || !grants.getFirst()) {
|
||||
if (grants.size() != 1) {
|
||||
throw unavailable();
|
||||
}
|
||||
if (grants.getFirst() == 1) {
|
||||
throw actionResponse ? unavailablePendingGrant() : unavailable();
|
||||
}
|
||||
if (grants.getFirst() != 2) {
|
||||
throw unavailable();
|
||||
}
|
||||
}
|
||||
@@ -338,6 +386,16 @@ public class WorkspaceExecutionStore {
|
||||
cause);
|
||||
}
|
||||
|
||||
// A creation grant or the registry state changes under operator
|
||||
// control: refusing a committed decision over it is not the
|
||||
// structural verdict of unavailable(), and the delivery machine must
|
||||
// retry so a restored grant still reaches the Harness. Structural
|
||||
// refusals keep unavailable().
|
||||
public static RuntimeBrokerException unavailablePendingGrant() {
|
||||
return new RuntimeBrokerException(409, "workspace_unavailable",
|
||||
"Workspace access grant or registry state is changing.", true);
|
||||
}
|
||||
|
||||
private static RuntimeBrokerException busy() {
|
||||
return new RuntimeBrokerException(409, "workspace_busy",
|
||||
"Workspace storage is held by another tool turn.", true);
|
||||
|
||||
+2
@@ -0,0 +1,2 @@
|
||||
-- Existing receipts retain their original requested-event boundary until retried.
|
||||
ALTER TABLE managed_agent_command ADD COLUMN mutation_attempt_sequence BIGINT;
|
||||
+4
-4
@@ -204,7 +204,7 @@
|
||||
"tags": ["Public Sessions"],
|
||||
"operationId": "createSession",
|
||||
"x-qwen-implementation-status": "implemented",
|
||||
"description": "Workspace selection is persisted with the original actor-scoped creation receipt. With the deployment's G0 file admission opt-in, qwen-code Sessions in an authorized, preregistered Workspace may include initial input under the fixed preapproved file-tool profile. Public callers cannot select the profile. Otherwise Workspace creation must have empty input. The Session creator may submit later Turns and cancel its running Turns under the same opt-in, and may rename the Session, while the creator currently holds Workspace read and create grants on a registry row whose state is ACTIVE and the Session is an active, undeleted qwen-code Session on the frozen execution profile; Workspace close follows its separate close capability and lifecycle admission; archive and unarchive follow retention capabilities after reliable Workspace close; delete supports either L3 idle ACTIVE files Sessions or L2 CLOSED/ARCHIVED Sessions; controlled same-Workspace cwd changes ship through the durable cwd_change operations (v1.32). This surface has no per-caller capability flag for later Turns, so callers must handle 409 workspace_unavailable. Check replay before resolving defaults; a durable binding is not proof of Runtime readiness.",
|
||||
"description": "Workspace selection is persisted with the original actor-scoped creation receipt. With the deployment's G0 file admission opt-in, qwen-code Sessions in an authorized, preregistered Workspace may include initial input under the fixed preapproved file-tool profile. Public callers cannot select the profile. Otherwise Workspace creation must have empty input. The Session creator may submit later Turns and rename the Session under the same opt-in, while the creator currently holds Workspace read and create grants on a registry row whose state is ACTIVE and still carries the Workspace generation and storage identity the Session was bound to, and the Session is an active, undeleted qwen-code Session on the frozen execution profile. Cancelling only aborts work that is already running, so the creator may cancel a running Turn under the same opt-in while they can still read the Workspace, even after create authority is revoked, the registry row leaves ACTIVE or the Workspace is re-registered (a live cancel reuses the owner's resident Harness attachment; a cold connector cache passively re-attaches for a persisted cancellation after frozen identity checks, independently of current creation grants, registry state or mount readiness; new work still rechecks execution authority); Workspace close follows its separate close capability and lifecycle admission; archive and unarchive follow retention capabilities after reliable Workspace close; delete supports either L3 idle ACTIVE files Sessions or L2 CLOSED/ARCHIVED Sessions; controlled same-Workspace cwd changes ship through the durable cwd_change operations (v1.32). This surface has no per-caller capability flag for later Turns, so callers must handle a refused later Turn: 409 workspace_unavailable, or 404 session_not_found when the caller no longer holds Workspace read. Check replay before resolving defaults; a durable binding is not proof of Runtime readiness.",
|
||||
"parameters": [
|
||||
{
|
||||
"$ref": "#/components/parameters/IdempotencyKey"
|
||||
@@ -1209,7 +1209,7 @@
|
||||
"tags": ["WebShell"],
|
||||
"operationId": "webShellCreateSession",
|
||||
"x-qwen-implementation-status": "implemented",
|
||||
"description": "Maps workspaceId/cwdRelative to public Workspace selection without using environmentId or absolute cwd. Shares G0's opt-in initial file-tool Turn admission and fixed server-owned profile with public Session creation. The Session creator may submit later Turns and cancel its running Turns under the same opt-in, and may rename the Session, while the creator currently holds Workspace read and create grants on a registry row whose state is ACTIVE and the Session is an active, undeleted qwen-code Session on the frozen execution profile; Workspace close follows its separate close capability and lifecycle admission; archive and unarchive follow retention capabilities after reliable Workspace close; delete supports either L3 idle ACTIVE files Sessions or L2 CLOSED/ARCHIVED Sessions; controlled same-Workspace cwd changes ship through the durable cwd_change operations (v1.32). The per-caller workspaceTurns capability on this surface advertises the same rule. Freeze selection with the original idempotency key; admission does not prove physical directory readiness.",
|
||||
"description": "Maps workspaceId/cwdRelative to public Workspace selection without using environmentId or absolute cwd. Shares G0's opt-in initial file-tool Turn admission and fixed server-owned profile with public Session creation. The Session creator may submit later Turns under the same opt-in with current read and create grants on the bound ACTIVE Workspace generation and storage. Cancellation of running work requires the creator’s read grant and the opt-in, even after create authority is revoked (a live cancel reuses the owner's resident Harness attachment; a cold connector cache passively re-attaches for a persisted cancellation after frozen identity checks, independently of current creation grants, registry state or mount readiness; new work still rechecks execution authority); Workspace close follows its separate close capability and lifecycle admission; archive and unarchive follow retention capabilities after reliable Workspace close; delete supports either L3 idle ACTIVE files Sessions or L2 CLOSED/ARCHIVED Sessions; controlled same-Workspace cwd changes ship through the durable cwd_change operations (v1.32). The per-caller workspaceTurns capability on this surface advertises current submit and rename admission; it does not gate cancellation. Freeze selection with the original idempotency key; admission does not prove physical directory readiness.",
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -5064,7 +5064,7 @@
|
||||
},
|
||||
"WebShellSessionCapabilities": {
|
||||
"type": "object",
|
||||
"required": ["tasks", "artifacts", "actions"],
|
||||
"required": ["tasks", "artifacts", "actions", "workspaceTurns"],
|
||||
"properties": {
|
||||
"workspaceContext": {
|
||||
"type": "boolean",
|
||||
@@ -5093,7 +5093,7 @@
|
||||
"workspaceTurns": {
|
||||
"type": "boolean",
|
||||
"default": false,
|
||||
"description": "True when the caller may submit later Turns of this Workspace-bound Session, cancel its running Turns and rename it: the deployment enables Workspace files, the caller created the Session and currently holds Workspace read and create grants on a registry row whose state is ACTIVE, and the Session is an active qwen-code Session on the frozen execution profile. False for every other caller and for unbound Sessions, which do not use it."
|
||||
"description": "True when the caller may submit later Turns of this Workspace-bound Session: the deployment enables Workspace files, the caller created the Session and currently holds Workspace read and create grants on a registry row whose state is ACTIVE and still carries the Workspace generation and storage identity the Session was bound to, and the Session is an active, undeleted qwen-code Session on the frozen execution profile. Cancelling only aborts work that is already running, so the creator may cancel a running Turn even when this is false, as long as they can still read the Workspace and the deployment still enables Workspace files. False for every other caller and for unbound Sessions, which do not use it."
|
||||
},
|
||||
"tasks": {
|
||||
"type": "boolean"
|
||||
|
||||
+61
-17
@@ -4,6 +4,7 @@ import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.awaitility.Awaitility.await;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.api.AuthenticatedTenantActor;
|
||||
import com.alibaba.qwen.code.managedagent.harness.HarnessConnector;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
@@ -47,6 +48,7 @@ import org.springframework.boot.web.servlet.FilterRegistrationBean;
|
||||
import org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext;
|
||||
import org.springframework.core.Ordered;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.test.util.ReflectionTestUtils;
|
||||
|
||||
class HostedPublicWorkspaceIT {
|
||||
private static final String TOKEN = "g0-local-fixture";
|
||||
@@ -438,12 +440,18 @@ class HostedPublicWorkspaceIT {
|
||||
List<Path> roots = List.of(Files.createDirectory(temporary.resolve("workspace-a")),
|
||||
Files.createDirectory(temporary.resolve("workspace-b")));
|
||||
for (Path root : roots) Files.createDirectory(root.resolve("child"));
|
||||
port = freePort();
|
||||
int harnessPort = freePort();
|
||||
int brokerPort = freePort();
|
||||
model = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
model.createContext("/v1/chat/completions", this::modelReply);
|
||||
model.start();
|
||||
int harnessPort;
|
||||
int brokerPort;
|
||||
try (ServerSocket springSocket = new ServerSocket(0);
|
||||
ServerSocket harnessSocket = new ServerSocket(0);
|
||||
ServerSocket brokerSocket = new ServerSocket(0)) {
|
||||
port = springSocket.getLocalPort();
|
||||
harnessPort = harnessSocket.getLocalPort();
|
||||
brokerPort = brokerSocket.getLocalPort();
|
||||
}
|
||||
startSpring(cli, roots, harnessPort, brokerPort);
|
||||
startHarness(cli, harnessPort, brokerPort);
|
||||
return roots;
|
||||
@@ -590,18 +598,33 @@ class HostedPublicWorkspaceIT {
|
||||
assertThat(request("PATCH", "/v1/agents/sessions/" + session, rename, "rename-" + workspace,
|
||||
"actor", 200).path("metadata").path("title").asText()).isEqualTo("Renamed " + workspace);
|
||||
|
||||
// A creator whose can_create grant is revoked keeps read access but loses
|
||||
// admission: submit, cancel and rename all answer workspace_unavailable,
|
||||
// nothing new executes, and no PENDING command row is left behind.
|
||||
// A creator whose can_create grant is revoked, with the Workspace draining, keeps
|
||||
// read access but loses admission of new work: submit and rename answer
|
||||
// workspace_unavailable, and no PENDING command row is left behind. Cancelling
|
||||
// only aborts work already running, so a Turn started before the revocation is
|
||||
// still cancelled and stops without running another tool.
|
||||
int beforeRevokedCancel = modelRequests.size();
|
||||
String revokedTurn = request("POST", "/v1/agents/sessions/" + session + "/events", hold,
|
||||
"hold-revoked-" + workspace, "actor", 202).path("turn_id").asText();
|
||||
await().atMost(Duration.ofSeconds(35)).until(() -> modelRequests.size() > beforeRevokedCancel);
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = FALSE"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ? AND actor_id = ?",
|
||||
tenant, workspace, "actor".getBytes(StandardCharsets.UTF_8));
|
||||
jdbc.update("UPDATE managed_workspace_registry SET state = 'DRAINING'"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ?", tenant, workspace);
|
||||
// Harness still runs the Turn, but this Java owner has lost its ref.
|
||||
((Map<?, ?>) ReflectionTestUtils.getField(spring.getBean(HarnessConnector.class), "attachments")).clear();
|
||||
Map<String, Object> revokedCancel = Map.of("type", "agent.session.cancel", "turn_id", revokedTurn);
|
||||
request("POST", "/v1/agents/sessions/" + session + "/events", revokedCancel,
|
||||
"nocreate-cancel-" + workspace, "actor", 202);
|
||||
await().atMost(Duration.ofSeconds(35)).untilAsserted(() -> assertThat(jdbc.queryForObject(
|
||||
"SELECT status FROM managed_agent_turn WHERE session_id = ? AND turn_id = ?",
|
||||
String.class, session, revokedTurn)).isEqualTo("CANCELLED"));
|
||||
heldReply.countDown();
|
||||
heldReply = new CountDownLatch(1);
|
||||
assertThat(request("POST", "/v1/agents/sessions/" + session + "/events", later,
|
||||
"nocreate-later-" + workspace, "actor", 409).path("error").path("code").asText())
|
||||
.isEqualTo("workspace_unavailable");
|
||||
assertThat(request("POST", "/v1/agents/sessions/" + session + "/events", cancel,
|
||||
"nocreate-cancel-" + workspace, "actor", 409).path("error").path("code").asText())
|
||||
.isEqualTo("workspace_unavailable");
|
||||
assertThat(request("PATCH", "/v1/agents/sessions/" + session, rename,
|
||||
"nocreate-rename-" + workspace, "actor", 409).path("error").path("code").asText())
|
||||
.isEqualTo("workspace_unavailable");
|
||||
@@ -612,6 +635,8 @@ class HostedPublicWorkspaceIT {
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = TRUE"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ? AND actor_id = ?",
|
||||
tenant, workspace, "actor".getBytes(StandardCharsets.UTF_8));
|
||||
jdbc.update("UPDATE managed_workspace_registry SET state = 'ACTIVE'"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ?", tenant, workspace);
|
||||
|
||||
// With the running Turn settled, revoking the creator's read grant hides the
|
||||
// bound Session from every later-Turn path: submit, cancel and rename all fall
|
||||
@@ -631,8 +656,27 @@ class HostedPublicWorkspaceIT {
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_read = TRUE"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ? AND actor_id = ?",
|
||||
tenant, workspace, "actor".getBytes(StandardCharsets.UTF_8));
|
||||
|
||||
String registeredStorage = jdbc.queryForObject("SELECT storage_id FROM managed_workspace_registry"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ?", String.class, tenant, workspace);
|
||||
jdbc.update("UPDATE managed_workspace_registry SET "
|
||||
+ (index == 0 ? "workspace_generation = workspace_generation + 1"
|
||||
: "storage_id = 'replacement-storage'")
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ?", tenant, workspace);
|
||||
assertThat(request("POST", "/api/agent/web-shell/v1/sessions/get", Map.of("sessionId", session),
|
||||
null, "actor", 200).path("capabilities").path("workspaceTurns").asBoolean()).isFalse();
|
||||
assertUnavailable(request("POST", "/v1/agents/sessions/" + session + "/events", later,
|
||||
"rebound-later-" + workspace, "actor", 409));
|
||||
assertUnavailable(request("PATCH", "/v1/agents/sessions/" + session, rename,
|
||||
"rebound-rename-" + workspace, "actor", 409));
|
||||
request("POST", "/v1/agents/sessions/" + session + "/events", revokedCancel,
|
||||
"rebound-cancel-" + workspace, "actor", 202);
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM managed_agent_command"
|
||||
+ " WHERE tenant_id = ? AND command_status = 'PENDING'", Integer.class, tenant)).isZero();
|
||||
jdbc.update("UPDATE managed_workspace_registry SET workspace_generation = 1, storage_id = ?"
|
||||
+ " WHERE tenant_id = ? AND workspace_id = ?", registeredStorage, tenant, workspace);
|
||||
}
|
||||
assertThat(modelRequests).hasSize(approvals ? 16 : 18);
|
||||
assertThat(modelRequests).hasSize(approvals ? 16 : 20);
|
||||
assertThat(modelFailure.get()).isNull();
|
||||
Map<String, Object> denied = Map.of("agent_id", "qwen-code", "workspace", Map.of("workspace_id", "workspace-0"),
|
||||
"input", List.of(Map.of("type", "input_text", "text", "G0_FILES")));
|
||||
@@ -665,7 +709,7 @@ class HostedPublicWorkspaceIT {
|
||||
assertUnavailable(request("POST", "/v1/agents/sessions", denied, "unsupported", "actor", 409));
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM managed_agent_session WHERE tenant_id = ?",
|
||||
Integer.class, tenant)).isEqualTo(2);
|
||||
assertThat(modelRequests).hasSize(approvals ? 16 : 18);
|
||||
assertThat(modelRequests).hasSize(approvals ? 16 : 20);
|
||||
}
|
||||
|
||||
private void startSpring(Path cli, List<Path> roots, int harnessPort, int brokerPort) {
|
||||
@@ -807,9 +851,13 @@ class HostedPublicWorkspaceIT {
|
||||
if (!harness.isAlive()) {
|
||||
throw new AssertionError("Hosted Harness exited: " + Files.readString(log));
|
||||
}
|
||||
assertThat(http.send(HttpRequest.newBuilder(URI.create("http://127.0.0.1:" + harnessPort + "/capabilities"))
|
||||
HttpResponse<String> response = http.send(HttpRequest.newBuilder(URI.create("http://127.0.0.1:" + harnessPort + "/capabilities"))
|
||||
.timeout(Duration.ofSeconds(2)).header("Authorization", "Bearer " + TOKEN).build(),
|
||||
HttpResponse.BodyHandlers.discarding()).statusCode()).isEqualTo(200);
|
||||
HttpResponse.BodyHandlers.ofString());
|
||||
assertThat(response.statusCode())
|
||||
.withFailMessage("Hosted Harness port %s returned %s: %s%n%s", harnessPort,
|
||||
response.statusCode(), response.body(), Files.readString(log))
|
||||
.isEqualTo(200);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -893,10 +941,6 @@ class HostedPublicWorkspaceIT {
|
||||
assertThat(refusal.path("error").path("code").asText()).isEqualTo("workspace_unavailable");
|
||||
}
|
||||
|
||||
private static int freePort() throws IOException {
|
||||
try (ServerSocket socket = new ServerSocket(0)) { return socket.getLocalPort(); }
|
||||
}
|
||||
|
||||
@AfterEach
|
||||
void stop() throws Exception {
|
||||
if (harness != null) {
|
||||
|
||||
+64
-1
@@ -23,8 +23,10 @@ import com.alibaba.qwen.code.managedagent.store.AgentStateStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedActionStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedSessionStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedSessionStoreModels.CommitTransactionRequest;
|
||||
import com.alibaba.qwen.code.managedagent.store.WorkspaceExecutionStore;
|
||||
import com.alibaba.qwen.code.runtimebroker.AesGcmSecretProtector;
|
||||
import com.alibaba.qwen.code.runtimebroker.JdbcRuntimeBindingRepository;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
@@ -567,7 +569,7 @@ class ManagedActionsTest {
|
||||
call -> {
|
||||
delivered.incrementAndGet();
|
||||
journal.change("decided", call.getArgument(3));
|
||||
throw new IllegalStateException("answer lost after commit");
|
||||
throw WorkspaceExecutionStore.unavailable();
|
||||
});
|
||||
mvc.perform(
|
||||
auth(post(path(session, journal.id) + "/responses"), tenant, "other")
|
||||
@@ -700,6 +702,67 @@ class ManagedActionsTest {
|
||||
.andExpect(status().isBadRequest());
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"permanent", "retryable", "other-code"})
|
||||
void workspaceAdmissionFailureFollowsRetryability(String failure) throws Exception {
|
||||
String tenant = tenant();
|
||||
String session = session(tenant);
|
||||
ActionJournal journal =
|
||||
action(tenant, session, System.currentTimeMillis(), 9007199254740991L);
|
||||
AtomicInteger attempts = new AtomicInteger();
|
||||
boolean terminal = "permanent".equals(failure);
|
||||
responses.put(journal.id, call -> {
|
||||
if (attempts.incrementAndGet() == 1) {
|
||||
throw switch (failure) {
|
||||
case "permanent" -> WorkspaceExecutionStore.unavailable();
|
||||
case "retryable" -> WorkspaceExecutionStore.unavailableTransient(
|
||||
new IllegalStateException("temporary authority failure"));
|
||||
default -> new RuntimeBrokerException(409, "runtime_unavailable",
|
||||
"Runtime authority is unavailable.", false);
|
||||
};
|
||||
}
|
||||
journal.change("decided", call.getArgument(3));
|
||||
return null;
|
||||
});
|
||||
try {
|
||||
JsonNode admitted = readAccepted(
|
||||
auth(post(path(session, journal.id) + "/responses"), tenant, "owner")
|
||||
.header("Idempotency-Key", "workspace-refusal")
|
||||
.contentType(MediaType.APPLICATION_JSON)
|
||||
.content(response("allow")));
|
||||
String op = admitted.path("id").asText();
|
||||
await().atMost(Duration.ofSeconds(5)).untilAsserted(() ->
|
||||
assertThat(sessions.findOperation(tenant, session, op).orElseThrow().state())
|
||||
.isEqualTo(terminal ? "FAILED" : "COMPLETED"));
|
||||
assertThat(attempts.get()).isEqualTo(terminal ? 1 : 2);
|
||||
assertThat(actions.find(tenant, session, journal.id).orElseThrow().state())
|
||||
.isEqualTo(terminal ? "requested" : "decided");
|
||||
assertThat(actions.response(tenant, session, op).errorCode())
|
||||
.isEqualTo(terminal ? "workspace_unavailable" : null);
|
||||
JsonNode result = read(auth(
|
||||
get("/v1/agents/sessions/{session}/operations/{op}", session, op), tenant, "owner"));
|
||||
assertThat(result.path("status").asText()).isEqualTo(terminal ? "failed" : "completed");
|
||||
if (terminal) {
|
||||
assertThat(result.path("failure_code").asText()).isEqualTo("workspace_unavailable");
|
||||
assertThat(result.has("action_resolution")).isFalse();
|
||||
}
|
||||
assertThat(OpenApiContract.load()
|
||||
.validate("/components/schemas/PublicCommandOperation", result)).isEmpty();
|
||||
JsonNode web = read(auth(post("/api/agent/web-shell/v1/operations/query"), tenant, "owner")
|
||||
.contentType(MediaType.APPLICATION_JSON)
|
||||
.content(json.writeValueAsString(Map.of("sessionId", session, "operationId", op))));
|
||||
assertThat(web.path("status").asText()).isEqualTo(terminal ? "failed" : "completed");
|
||||
if (terminal) {
|
||||
assertThat(web.path("failureCode").asText()).isEqualTo("workspace_unavailable");
|
||||
assertThat(web.has("actionResolution")).isFalse();
|
||||
}
|
||||
assertThat(OpenApiContract.load()
|
||||
.validate("/components/schemas/WebShellCommandOperation", web)).isEmpty();
|
||||
} finally {
|
||||
responses.remove(journal.id);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void recoveryBlockedResponsesStayRunningUntilJournalProvesTheOutcome() throws Exception {
|
||||
String tenant = tenant();
|
||||
|
||||
+3
-1
@@ -35,6 +35,7 @@ import com.alibaba.qwen.code.managedagent.store.StoreModels.ItemRecord;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.OperationKind;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.ProjectedEvent;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.SessionMutationKind;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import java.time.Clock;
|
||||
@@ -1955,7 +1956,8 @@ class ManagedAgentServerIntegrationTest {
|
||||
renames.incrementAndGet();
|
||||
if (renameFailures.getAndUpdate(value -> Math.max(0,
|
||||
value - 1)) > 0) {
|
||||
throw new IllegalStateException("fixture rename failure");
|
||||
throw new RuntimeBrokerException(503, "fixture_rename_unavailable",
|
||||
"fixture rename failure", true);
|
||||
}
|
||||
titles.put(sessionId, title);
|
||||
}
|
||||
|
||||
+133
-2
@@ -498,6 +498,134 @@ class ManagedSessionLifecycleTest {
|
||||
Integer.class, tenant, sessionId)).isEqualTo(1);
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void retiredRenameCannotCompleteOverALaterCompletedRename(boolean legacyReceipt)
|
||||
throws Exception {
|
||||
String tenant = tenant();
|
||||
String sessionId = attachedSession(tenant);
|
||||
String bootId = store.requireSession(tenant, sessionId).harnessBootId();
|
||||
RequestDigests digests = new RequestDigests();
|
||||
String first = digests.digest(
|
||||
java.util.Map.of("sessionId", sessionId, "title", "A"));
|
||||
String second = digests.digest(
|
||||
java.util.Map.of("sessionId", sessionId, "title", "B"));
|
||||
// K1 and its same-key retry both enter the Harness; one fails and
|
||||
// retires K1, which frees the Session for a fresh key.
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", first,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", first,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.abandonSessionMutation(tenant, "RENAME_SESSION", "k1", sessionId);
|
||||
if (legacyReceipt) {
|
||||
assertThat(jdbc.update("UPDATE managed_agent_command SET mutation_attempt_sequence"
|
||||
+ " = NULL WHERE tenant_id = ? AND operation = ?"
|
||||
+ " AND idempotency_key = ?",
|
||||
tenant, "RENAME_SESSION", "k1")).isEqualTo(1);
|
||||
}
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k2", second,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.completeSessionMutation(tenant, "RENAME_SESSION", "k2", sessionId,
|
||||
SessionMutationKind.RENAME, "B", bootId);
|
||||
|
||||
// The sibling still inside the Harness finishes K1 last: it must
|
||||
// not revert the newer committed title.
|
||||
assertThatThrownBy(() -> store.completeSessionMutation(tenant,
|
||||
"RENAME_SESSION", "k1", sessionId, SessionMutationKind.RENAME,
|
||||
"A", bootId))
|
||||
.isInstanceOfSatisfying(ApiException.class, error ->
|
||||
assertThat(error.getCode())
|
||||
.isEqualTo("session_mutation_superseded"));
|
||||
assertThat(store.requireSession(tenant, sessionId).title()).isEqualTo("B");
|
||||
assertThat(jdbc.queryForObject("SELECT command_status FROM"
|
||||
+ " managed_agent_command WHERE tenant_id = ? AND"
|
||||
+ " operation = 'RENAME_SESSION' AND idempotency_key = 'k1'",
|
||||
String.class, tenant)).isEqualTo("FAILED");
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM managed_agent_event"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND event_type = 'session.updated'",
|
||||
Integer.class, tenant, sessionId)).isEqualTo(1);
|
||||
|
||||
// A same-key request sent after K2 is the newest request, so it
|
||||
// still re-drives K1 and wins.
|
||||
lifecycle(patch("/v1/agents/sessions/{id}", sessionId)
|
||||
.contentType(MediaType.APPLICATION_JSON)
|
||||
.content("{\"title\":\"A\"}"), tenant, "k1")
|
||||
.andExpect(status().isOk())
|
||||
.andExpect(jsonPath("$.metadata.title").value("A"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void pendingRenameCompletesAfterAnOlderRetiredSibling() throws Exception {
|
||||
String tenant = tenant();
|
||||
String sessionId = attachedSession(tenant);
|
||||
String bootId = store.requireSession(tenant, sessionId).harnessBootId();
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", "first",
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", "first",
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.abandonSessionMutation(tenant, "RENAME_SESSION", "k1", sessionId);
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k2", "second",
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
assertThat(store.completeSessionMutation(tenant, "RENAME_SESSION", "k1",
|
||||
sessionId, SessionMutationKind.RENAME, "A", bootId).title())
|
||||
.isEqualTo("A");
|
||||
assertThat(store.completeSessionMutation(tenant, "RENAME_SESSION", "k2",
|
||||
sessionId, SessionMutationKind.RENAME, "B", bootId).title())
|
||||
.isEqualTo("B");
|
||||
assertThat(store.requireSession(tenant, sessionId).title()).isEqualTo("B");
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void latestRenameAttemptCompletesAfterItsSiblingRetires(boolean recreatedReceipt)
|
||||
throws Exception {
|
||||
String tenant = tenant();
|
||||
String sessionId = attachedSession(tenant);
|
||||
String bootId = store.requireSession(tenant, sessionId).harnessBootId();
|
||||
RequestDigests digests = new RequestDigests();
|
||||
String first = digests.digest(
|
||||
java.util.Map.of("sessionId", sessionId, "title", "A"));
|
||||
String second = digests.digest(
|
||||
java.util.Map.of("sessionId", sessionId, "title", "B"));
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", first,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.abandonSessionMutation(tenant, "RENAME_SESSION", "k1", sessionId);
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k2", second,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
harness.rename(tenant, sessionId, "B");
|
||||
store.completeSessionMutation(tenant, "RENAME_SESSION", "k2", sessionId,
|
||||
SessionMutationKind.RENAME, "B", bootId);
|
||||
if (recreatedReceipt) {
|
||||
assertThat(jdbc.update("DELETE FROM managed_agent_command WHERE tenant_id = ?"
|
||||
+ " AND operation = ? AND idempotency_key = ?",
|
||||
tenant, "RENAME_SESSION", "k1")).isEqualTo(1);
|
||||
}
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", first,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.beginSessionMutation(tenant, "RENAME_SESSION", "k1", first,
|
||||
sessionId, SessionMutationKind.RENAME);
|
||||
store.abandonSessionMutation(tenant, "RENAME_SESSION", "k1", sessionId);
|
||||
harness.rename(tenant, sessionId, "A");
|
||||
assertThat(store.completeSessionMutation(tenant, "RENAME_SESSION", "k1",
|
||||
sessionId, SessionMutationKind.RENAME, "A", bootId).title())
|
||||
.isEqualTo("A");
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM managed_agent_event"
|
||||
+ " WHERE tenant_id = ? AND session_id = ? AND source_key = ?",
|
||||
Integer.class, tenant, sessionId,
|
||||
"control:RENAME_SESSION:k1:requested")).isEqualTo(1);
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM managed_agent_event"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND event_type = 'session.updated'",
|
||||
Integer.class, tenant, sessionId)).isEqualTo(2);
|
||||
lifecycle(patch("/v1/agents/sessions/{id}", sessionId)
|
||||
.contentType(MediaType.APPLICATION_JSON)
|
||||
.content("{\"title\":\"A\"}"), tenant, "k1")
|
||||
.andExpect(status().isOk())
|
||||
.andExpect(jsonPath("$.metadata.title").value("A"))
|
||||
.andExpect(header().string("X-Qwen-Idempotent-Replay", "true"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryingFailedRenameAgainBlocksOtherLifecycleWork() throws Exception {
|
||||
String tenant = tenant();
|
||||
@@ -522,9 +650,12 @@ class ManagedSessionLifecycleTest {
|
||||
String sessionId = attachedSession(tenant);
|
||||
AgentStateStore faulted = mock(AgentStateStore.class, delegatesTo(store));
|
||||
if ("lookup".equals(phase)) {
|
||||
AtomicInteger reads = new AtomicInteger();
|
||||
doAnswer(call -> {
|
||||
if (reads.incrementAndGet() == 2) {
|
||||
if (jdbc.queryForObject("SELECT COUNT(*) FROM managed_agent_command"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND operation = 'RENAME_SESSION' AND idempotency_key = 'key'"
|
||||
+ " AND command_status = 'PENDING'", Integer.class,
|
||||
tenant, sessionId) > 0) {
|
||||
throw new IllegalStateException("lookup unavailable");
|
||||
}
|
||||
return store.requireSession(tenant, sessionId);
|
||||
|
||||
+255
@@ -12,13 +12,16 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
|
||||
|
||||
import com.alibaba.qwen.code.daemon.DaemonHttpException;
|
||||
import com.alibaba.qwen.code.managedagent.api.ApiException;
|
||||
import com.alibaba.qwen.code.managedagent.api.ApiModels.CommandAdmission;
|
||||
import com.alibaba.qwen.code.managedagent.api.ApiModels.InputBlock;
|
||||
import com.alibaba.qwen.code.managedagent.api.AuthenticatedTenantActor;
|
||||
import com.alibaba.qwen.code.managedagent.api.TenantContextFilter;
|
||||
import com.alibaba.qwen.code.managedagent.api.WorkspaceSelection;
|
||||
import com.alibaba.qwen.code.managedagent.config.ManagedAgentProperties;
|
||||
import com.alibaba.qwen.code.managedagent.harness.HarnessConnector;
|
||||
import com.alibaba.qwen.code.managedagent.harness.HarnessConnector.Attachment;
|
||||
import com.alibaba.qwen.code.managedagent.harness.UnavailableHarnessConnector;
|
||||
import com.alibaba.qwen.code.managedagent.service.HarnessCoordinator;
|
||||
import com.alibaba.qwen.code.managedagent.service.ManagedAgentService;
|
||||
import com.alibaba.qwen.code.managedagent.service.RequestDigests;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedAgentStore;
|
||||
@@ -35,6 +38,7 @@ import java.util.UUID;
|
||||
import java.util.concurrent.CyclicBarrier;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import org.assertj.core.api.ThrowableAssert.ThrowingCallable;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
@@ -1167,6 +1171,257 @@ class ManagedWorkspaceAdmissionTest {
|
||||
+ " WHERE tenant_id = ?", Integer.class, tenant)).isZero();
|
||||
}
|
||||
|
||||
@Test
|
||||
void creatorCancelsWithoutTheGrantsThatAdmitNewWork() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
grant(tenant, "ws-a", "actor-b", false);
|
||||
ManagedAgentService enabled = boundService(true);
|
||||
ManagedAgentService optedOut = boundService(false);
|
||||
assertThat(enabled.getWebShellSession(tenant, "actor-a", sessionId)
|
||||
.capabilities().workspaceTurns()).isTrue();
|
||||
// The opt-in clause: the same creator and Session without it.
|
||||
assertThat(optedOut.getWebShellSession(tenant, "actor-a", sessionId)
|
||||
.capabilities().workspaceTurns()).isFalse();
|
||||
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = FALSE"
|
||||
+ " WHERE tenant_id = ?", tenant);
|
||||
jdbc.update("UPDATE managed_workspace_registry SET state = 'DRAINING'"
|
||||
+ " WHERE tenant_id = ?", tenant);
|
||||
assertThat(enabled.getWebShellSession(tenant, "actor-a", sessionId)
|
||||
.capabilities().workspaceTurns()).isFalse();
|
||||
assertRefused(() -> enabled.submitTurn(tenant, "actor-a", "submit",
|
||||
sessionId, List.of(new InputBlock("text", "go"))),
|
||||
"workspace_unavailable");
|
||||
// Cancel admission passes for the creator; the refusal comes from the
|
||||
// missing Turn, after admission.
|
||||
assertRefused(() -> enabled.cancelTurn(tenant, "actor-a", "cancel",
|
||||
sessionId, "turn_missing"), "turn_not_found");
|
||||
// A reader who did not create the Session keeps the refusal, and so
|
||||
// does the creator without the opt-in.
|
||||
assertRefused(() -> enabled.cancelTurn(tenant, "actor-b", "cancel-b",
|
||||
sessionId, "turn_missing"), "workspace_unavailable");
|
||||
assertRefused(() -> optedOut.cancelTurn(tenant, "actor-a",
|
||||
"cancel-off", sessionId, "turn_missing"),
|
||||
"workspace_unavailable");
|
||||
}
|
||||
|
||||
@Test
|
||||
void cancelRefusesABoundSessionThatCanNoLongerExecute() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
ManagedAgentService service = boundServiceWithWorkingHarness();
|
||||
String turnId = service.submitTurn(tenant, "actor-a", "submit-1",
|
||||
sessionId, List.of(new InputBlock("text", "go"))).turnId();
|
||||
|
||||
// Cancelling cites the admission shape, so closing the Session
|
||||
// restores the legacy refusal instead of writing a CANCEL command
|
||||
// for a Turn that can no longer run.
|
||||
jdbc.update("UPDATE managed_agent_session SET status = 'CLOSED'"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?",
|
||||
tenant, sessionId);
|
||||
assertRefused(() -> service.cancelTurn(tenant, "actor-a", "cancel-1",
|
||||
sessionId, turnId), "workspace_unavailable");
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM"
|
||||
+ " managed_agent_command WHERE tenant_id = ?"
|
||||
+ " AND operation = 'CANCEL_TURN'", Integer.class, tenant))
|
||||
.isZero();
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void reRegistrationRefusesLaterWorkBeforeAnyCommandIsWritten(boolean storageOnly) {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
ManagedAgentService enabled = boundService(true);
|
||||
jdbc.update("UPDATE managed_workspace_registry SET"
|
||||
+ (storageOnly ? " storage_id = 'replacement-storage'"
|
||||
: " workspace_generation = workspace_generation + 1")
|
||||
+ " WHERE tenant_id = ?", tenant);
|
||||
|
||||
assertThat(enabled.getWebShellSession(tenant, "actor-a", sessionId)
|
||||
.capabilities().workspaceTurns()).isFalse();
|
||||
assertRefused(() -> enabled.submitTurn(tenant, "actor-a", "submit",
|
||||
sessionId, List.of(new InputBlock("text", "go"))),
|
||||
"workspace_unavailable");
|
||||
assertRefused(() -> enabled.renameSession(tenant, "actor-a",
|
||||
"rename", sessionId, "Renamed"), "workspace_unavailable");
|
||||
// The rename was refused before its command was written, so no
|
||||
// PENDING command blocks a later operation.
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM"
|
||||
+ " managed_agent_command WHERE tenant_id = ? AND"
|
||||
+ " command_status = 'PENDING'", Integer.class, tenant))
|
||||
.isZero();
|
||||
assertRefused(() -> enabled.cancelTurn(tenant, "actor-a", "cancel",
|
||||
sessionId, "turn_missing"), "turn_not_found");
|
||||
}
|
||||
|
||||
@Test
|
||||
void sameKeySubmitReplayRequiresWorkspaceReadAccess() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
ManagedAgentService service = boundServiceWithWorkingHarness();
|
||||
List<InputBlock> input = List.of(new InputBlock("text", "go"));
|
||||
CommandAdmission first = service.submitTurn(tenant, "actor-a",
|
||||
"submit-1", sessionId, input);
|
||||
assertThat(first.replayed()).isFalse();
|
||||
assertThat(first.turnId()).isNotBlank();
|
||||
|
||||
assertRefused(() -> service.submitTurn(tenant, "actor-b",
|
||||
"submit-1", sessionId, input), "session_not_found");
|
||||
}
|
||||
|
||||
@Test
|
||||
void sameKeyReplayAnswersOnlyTheCreator() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
grant(tenant, "ws-a", "actor-b", false);
|
||||
ManagedAgentService service = boundServiceWithWorkingHarness();
|
||||
List<InputBlock> input = List.of(new InputBlock("text", "go"));
|
||||
service.submitTurn(tenant, "actor-a", "submit-1", sessionId, input);
|
||||
service.renameSession(tenant, "actor-a", "rename-1", sessionId,
|
||||
"renamed title");
|
||||
|
||||
// A reader who did not create the Session neither replays the
|
||||
// creator's admissions nor learns which keys exist.
|
||||
assertRefused(() -> service.submitTurn(tenant, "actor-b",
|
||||
"submit-1", sessionId, input), "workspace_unavailable");
|
||||
assertRefused(() -> service.submitTurn(tenant, "actor-b",
|
||||
"submit-1", sessionId,
|
||||
List.of(new InputBlock("text", "other"))),
|
||||
"workspace_unavailable");
|
||||
assertRefused(() -> service.renameSession(tenant, "actor-b",
|
||||
"rename-1", sessionId, "renamed title"),
|
||||
"workspace_unavailable");
|
||||
}
|
||||
|
||||
@Test
|
||||
void sameKeySubmitReplaysTheRecordedAdmissionAfterReRegistration() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
ManagedAgentService service = boundServiceWithWorkingHarness();
|
||||
|
||||
CommandAdmission first = service.submitTurn(tenant, "actor-a",
|
||||
"submit-1", sessionId,
|
||||
List.of(new InputBlock("text", "go")));
|
||||
assertThat(first.replayed()).isFalse();
|
||||
assertThat(first.turnId()).isNotBlank();
|
||||
|
||||
// Re-registration flips the admission gate off; the recorded
|
||||
// admission must still answer the same-key retry — replay is a read
|
||||
// of the record, not a re-admission.
|
||||
jdbc.update("UPDATE managed_workspace_registry SET"
|
||||
+ " workspace_generation = workspace_generation + 1"
|
||||
+ " WHERE tenant_id = ?", tenant);
|
||||
assertThat(service.getWebShellSession(tenant, "actor-a", sessionId)
|
||||
.capabilities().workspaceTurns()).isFalse();
|
||||
|
||||
CommandAdmission second = service.submitTurn(tenant, "actor-a",
|
||||
"submit-1", sessionId,
|
||||
List.of(new InputBlock("text", "go")));
|
||||
assertThat(second.replayed()).isTrue();
|
||||
assertThat(second.turnId()).isEqualTo(first.turnId());
|
||||
}
|
||||
|
||||
@Test
|
||||
void sameKeyRenameReplaysTheRecordedOutcomeAfterReRegistration() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String sessionId = boundSession(tenant);
|
||||
ManagedAgentService service = boundServiceWithWorkingHarness();
|
||||
|
||||
var first = service.renameSession(tenant, "actor-a", "rename-1",
|
||||
sessionId, "renamed title");
|
||||
assertThat(first.replayed()).isFalse();
|
||||
assertThat(first.body().metadata())
|
||||
.containsEntry("title", "renamed title");
|
||||
|
||||
jdbc.update("UPDATE managed_workspace_registry SET"
|
||||
+ " workspace_generation = workspace_generation + 1"
|
||||
+ " WHERE tenant_id = ?", tenant);
|
||||
assertRefused(() -> service.renameSession(tenant, "actor-a",
|
||||
"rename-fresh", sessionId, "other"), "workspace_unavailable");
|
||||
|
||||
var second = service.renameSession(tenant, "actor-a", "rename-1",
|
||||
sessionId, "renamed title");
|
||||
assertThat(second.replayed()).isTrue();
|
||||
assertThat(second.body().metadata())
|
||||
.containsEntry("title", "renamed title");
|
||||
}
|
||||
|
||||
private ManagedAgentService boundServiceWithWorkingHarness() {
|
||||
return boundServiceWith(new UnavailableHarnessConnector() {
|
||||
@Override
|
||||
public boolean isAvailable() {
|
||||
return true;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isWorkspaceFilesAvailable() {
|
||||
return true;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Attachment createOrLoad(String tenantId, String sessionId,
|
||||
boolean loadExisting) {
|
||||
return new Attachment("boot");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void rename(String tenantId, String sessionId,
|
||||
String title) {
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private ManagedAgentService boundServiceWith(HarnessConnector harness) {
|
||||
ManagedAgentProperties properties = new ManagedAgentProperties();
|
||||
properties.getHarness().setWorkspaceFilesEnabled(true);
|
||||
ManagedAgentStore gated = new ManagedAgentStore(jdbc, mapper,
|
||||
Clock.systemUTC(), ignored -> {
|
||||
}, registry, properties);
|
||||
HarnessCoordinator noopCoordinator = new HarnessCoordinator(null,
|
||||
null, null, null, null, Clock.systemUTC(),
|
||||
new ManagedAgentProperties()) {
|
||||
@Override
|
||||
public void dispatch(String tenantId, String sessionId,
|
||||
String turnId) {
|
||||
}
|
||||
};
|
||||
return new ManagedAgentService(gated, new RequestDigests(),
|
||||
noopCoordinator, harness, registry);
|
||||
}
|
||||
|
||||
private String boundSession(String tenant) {
|
||||
register(tenant, "ws-a", "storage-a",
|
||||
WorkspaceExecutionProfile.CONFIG_REF,
|
||||
WorkspaceExecutionProfile.POLICY_REF);
|
||||
grant(tenant, "ws-a", "actor-a", true);
|
||||
return store.insertWorkspaceSessionCommand(tenant, "actor-a",
|
||||
"create", "sha256:" + "a".repeat(64), "qwen-code", null, null,
|
||||
List.of(), null, new WorkspaceSelection("ws-a", "."))
|
||||
.sessionId();
|
||||
}
|
||||
|
||||
private ManagedAgentService boundService(boolean workspaceFiles) {
|
||||
ManagedAgentProperties properties = new ManagedAgentProperties();
|
||||
properties.getHarness().setWorkspaceFilesEnabled(workspaceFiles);
|
||||
ManagedAgentStore gated = new ManagedAgentStore(jdbc, mapper,
|
||||
Clock.systemUTC(), ignored -> {
|
||||
}, registry, properties);
|
||||
return new ManagedAgentService(gated, new RequestDigests(), null,
|
||||
new UnavailableHarnessConnector() {
|
||||
@Override
|
||||
public boolean isWorkspaceFilesAvailable() {
|
||||
return workspaceFiles;
|
||||
}
|
||||
}, registry);
|
||||
}
|
||||
|
||||
private static void assertRefused(ThrowingCallable call, String code) {
|
||||
assertThatThrownBy(call).isInstanceOfSatisfying(ApiException.class,
|
||||
error -> assertThat(error.getCode()).isEqualTo(code));
|
||||
}
|
||||
|
||||
private void assertCreateError(String tenant, WorkspaceSelection selection,
|
||||
String expected) {
|
||||
assertThatThrownBy(() -> store.insertWorkspaceSessionCommand(tenant,
|
||||
|
||||
+185
@@ -0,0 +1,185 @@
|
||||
package com.alibaba.qwen.code.managedagent.harness;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.Mockito.clearInvocations;
|
||||
import static org.mockito.Mockito.doAnswer;
|
||||
import static org.mockito.Mockito.doNothing;
|
||||
import static org.mockito.Mockito.doThrow;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.never;
|
||||
import static org.mockito.Mockito.verify;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
import com.alibaba.qwen.code.daemon.HarnessSessionRef;
|
||||
import com.alibaba.qwen.code.daemon.HarnessRuntimeRecovery;
|
||||
import com.alibaba.qwen.code.daemon.HostedHarnessCapabilities;
|
||||
import com.alibaba.qwen.code.daemon.HostedHarnessClient;
|
||||
import com.alibaba.qwen.code.managedagent.api.ApiException;
|
||||
import com.alibaba.qwen.code.managedagent.api.WorkspaceSelection;
|
||||
import com.alibaba.qwen.code.managedagent.config.ManagedAgentProperties;
|
||||
import com.alibaba.qwen.code.managedagent.service.HarnessCoordinator;
|
||||
import com.alibaba.qwen.code.managedagent.service.HarnessEventProjector;
|
||||
import com.alibaba.qwen.code.managedagent.service.ManagedAgentService;
|
||||
import com.alibaba.qwen.code.managedagent.service.RequestDigests;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedActionStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedAgentStore;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedWorkspaceRegistry;
|
||||
import com.alibaba.qwen.code.managedagent.store.WorkspaceExecutionStore;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.test.util.ReflectionTestUtils;
|
||||
import org.springframework.transaction.PlatformTransactionManager;
|
||||
import org.springframework.transaction.annotation.Transactional;
|
||||
|
||||
@SpringBootTest(properties = {
|
||||
"spring.datasource.url=jdbc:h2:mem:cold-cancel;MODE=MySQL;"
|
||||
+ "DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE",
|
||||
"spring.datasource.driver-class-name=org.h2.Driver",
|
||||
"spring.datasource.username=sa",
|
||||
"spring.datasource.password=",
|
||||
"qwen.managed-agent.harness.enabled=false"
|
||||
})
|
||||
class QwenHostedHarnessColdCancelRegressionTest {
|
||||
@Autowired
|
||||
private JdbcTemplate jdbc;
|
||||
|
||||
@Autowired
|
||||
private ObjectMapper mapper;
|
||||
|
||||
@Autowired
|
||||
private ManagedWorkspaceRegistry registry;
|
||||
|
||||
@Autowired
|
||||
private PlatformTransactionManager transactionManager;
|
||||
|
||||
@Test
|
||||
@Transactional
|
||||
void coldCancellationRequiresPersistedIntentAndSurvivesMutableAuthorityChanges() {
|
||||
String tenant = "tenant-" + UUID.randomUUID();
|
||||
String boot = "11111111-1111-4111-8111-111111111111";
|
||||
jdbc.update("INSERT INTO managed_workspace_registry (tenant_id, workspace_id,"
|
||||
+ " workspace_generation, storage_id, display_name, config_ref, policy_ref, state)"
|
||||
+ " VALUES (?, 'ws-a', 1, 'storage-a', 'Workspace', ?, ?, 'ACTIVE')",
|
||||
tenant, WorkspaceExecutionProfile.CONFIG_REF, WorkspaceExecutionProfile.POLICY_REF);
|
||||
jdbc.update("INSERT INTO managed_workspace_access (tenant_id, workspace_id, actor_id, can_read, can_create)"
|
||||
+ " VALUES (?, 'ws-a', ?, TRUE, TRUE)",
|
||||
tenant, "actor-a".getBytes(StandardCharsets.UTF_8));
|
||||
ManagedAgentProperties properties = new ManagedAgentProperties();
|
||||
properties.getHarness().setWorkspaceFilesEnabled(true);
|
||||
properties.getHarness().setToken("test-token");
|
||||
properties.getHarness().setCapabilityDigest("sha256:" + "a".repeat(64));
|
||||
ManagedAgentStore store = new ManagedAgentStore(jdbc, mapper, Clock.systemUTC(),
|
||||
ignored -> { }, registry, properties);
|
||||
var admission = store.insertWorkspaceSessionCommand(tenant, "actor-a", "create", "digest",
|
||||
"qwen-code", null, null, List.of(), null, new WorkspaceSelection("ws-a", "."));
|
||||
String session = admission.sessionId();
|
||||
String turn = store.insertTurnCommand(tenant, "SUBMIT", "submit", "digest", session,
|
||||
List.of(Map.of("type", "text", "text", "go")), "payload").turnId();
|
||||
assertThat(turn).isNotNull();
|
||||
|
||||
HostedHarnessClient client = mock(HostedHarnessClient.class);
|
||||
HostedHarnessCapabilities capabilities = mock(HostedHarnessCapabilities.class);
|
||||
HarnessSessionRef attached = mock(HarnessSessionRef.class);
|
||||
when(client.capabilities()).thenReturn(capabilities);
|
||||
when(capabilities.getBootId()).thenReturn(boot);
|
||||
when(client.loadSession(any())).thenReturn(attached);
|
||||
when(attached.getHarnessBootId()).thenReturn(boot);
|
||||
when(attached.getApprovalMode()).thenReturn("yolo");
|
||||
HarnessRuntimeRecovery recovery = mock(HarnessRuntimeRecovery.class);
|
||||
when(recovery.getCheckpointId()).thenReturn("checkpoint-of-T1");
|
||||
when(attached.getRuntimeRecovery()).thenReturn(recovery);
|
||||
ManagedActionStore actions = mock(ManagedActionStore.class);
|
||||
when(actions.approvalMode(tenant, session)).thenReturn("yolo");
|
||||
WorkspaceExecutionStore execution = new WorkspaceExecutionStore(jdbc, transactionManager);
|
||||
QwenHostedHarnessConnector connector = new QwenHostedHarnessConnector(properties, store, execution, actions);
|
||||
ReflectionTestUtils.setField(connector, "client", client);
|
||||
connector.createOrLoad(tenant, session, true);
|
||||
|
||||
ExecutorService executor = mock(ExecutorService.class);
|
||||
doAnswer(invocation -> {
|
||||
invocation.<Runnable>getArgument(0).run();
|
||||
return null;
|
||||
}).when(executor).execute(any(Runnable.class));
|
||||
HarnessCoordinator coordinator = new HarnessCoordinator(store, connector,
|
||||
new HarnessEventProjector(), null, executor, Clock.systemUTC(), properties) {
|
||||
@Override
|
||||
public void dispatch(String tenantId, String sessionId, String turnId) {
|
||||
}
|
||||
};
|
||||
try {
|
||||
String owner = (String) ReflectionTestUtils.getField(coordinator, "owner");
|
||||
assertThat(store.claimTurn(tenant, session, turn, owner, Duration.ofMinutes(1))).isPresent();
|
||||
assertThat(store.bindHarness(tenant, session, turn, owner, boot)).isTrue();
|
||||
store.markSubmissionAttempted(tenant, session, turn, owner);
|
||||
store.recordAdmission(tenant, session, turn, owner, "epoch", 1);
|
||||
assertThatThrownBy(() -> connector.recoverManagedRuntime(tenant, session, true))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class,
|
||||
error -> assertThat(error.getCode()).isEqualTo("workspace_unavailable"));
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = FALSE WHERE tenant_id = ?", tenant);
|
||||
assertThat(store.insertCancelCommand(tenant, "CANCEL", "cancel", "digest", session, turn)
|
||||
.commandEffect()).isTrue();
|
||||
|
||||
coordinator.cancel(tenant, session, turn);
|
||||
verify(client).cancelTurn(attached);
|
||||
clearInvocations(client);
|
||||
((Map<?, ?>) ReflectionTestUtils.getField(connector, "attachments")).clear();
|
||||
|
||||
doThrow(new IllegalStateException("temporary cancel failure"))
|
||||
.when(client).cancelTurn(attached);
|
||||
assertThatThrownBy(() -> connector.cancel(tenant, session))
|
||||
.isInstanceOf(IllegalStateException.class);
|
||||
assertThat(connector.recoverManagedRuntime(tenant, session, true).runtimeRecovery())
|
||||
.isSameAs(recovery);
|
||||
clearInvocations(client);
|
||||
doNothing().when(client).cancelTurn(attached);
|
||||
coordinator.cancel(tenant, session, turn);
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = TRUE WHERE tenant_id = ?", tenant);
|
||||
assertThat(connector.recoverManagedRuntime(tenant, session, false).runtimeRecovery()).isNull();
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = FALSE WHERE tenant_id = ?", tenant);
|
||||
|
||||
assertThat(store.findTurn(tenant, session, turn).orElseThrow().status()).isEqualTo("CANCELLING");
|
||||
verify(client).cancelTurn(attached);
|
||||
assertThat(execution.verifiedRecoveryEnabled()).isFalse();
|
||||
assertThatThrownBy(() -> connector.submit(tenant, session, "later", List.of(), "digest"))
|
||||
.isInstanceOf(RuntimeBrokerException.class);
|
||||
verify(client, never()).submitTurn(any());
|
||||
|
||||
for (String change : List.of(
|
||||
"UPDATE managed_workspace_access SET can_read = FALSE WHERE tenant_id = ?",
|
||||
"UPDATE managed_workspace_registry SET state = 'DRAINING' WHERE tenant_id = ?",
|
||||
"UPDATE managed_workspace_registry SET workspace_generation = workspace_generation + 1 WHERE tenant_id = ?",
|
||||
"UPDATE managed_workspace_registry SET storage_id = 'replacement-storage' WHERE tenant_id = ?")) {
|
||||
jdbc.update(change, tenant);
|
||||
clearInvocations(client);
|
||||
((Map<?, ?>) ReflectionTestUtils.getField(connector, "attachments")).clear();
|
||||
coordinator.cancel(tenant, session, turn);
|
||||
verify(client).cancelTurn(attached);
|
||||
}
|
||||
|
||||
ManagedAgentService service = new ManagedAgentService(store, new RequestDigests(),
|
||||
coordinator, connector, registry);
|
||||
clearInvocations(client);
|
||||
assertThatThrownBy(() -> service.cancelTurn(tenant, "actor-a", "cancel-fresh", session, turn))
|
||||
.isInstanceOfSatisfying(ApiException.class,
|
||||
error -> assertThat(error.getCode()).isEqualTo("session_not_found"));
|
||||
assertThat(store.findCommand(tenant, "CANCEL", "cancel-fresh")).isEmpty();
|
||||
} finally {
|
||||
coordinator.close();
|
||||
connector.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
+10
-7
@@ -403,15 +403,16 @@ class QwenHostedHarnessConnectorTest {
|
||||
verify(client).createSession(any(CreateHarnessSession.class));
|
||||
}
|
||||
|
||||
@Test
|
||||
void rechecksWorkspaceAuthorityOnCachedAttachmentAndKeepsPassiveRecoveryAuthorized() {
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void rechecksWorkspaceAuthorityOnCachedAttachmentAndKeepsPassiveRecoveryAuthorized(boolean verifiedRecovery) {
|
||||
HostedHarnessClient client = mock(HostedHarnessClient.class);
|
||||
HostedHarnessCapabilities capabilities = mock(HostedHarnessCapabilities.class);
|
||||
HarnessSessionRef attached = mock(HarnessSessionRef.class);
|
||||
SessionRecord session = mock(SessionRecord.class);
|
||||
AgentStateStore sessions = mock(AgentStateStore.class);
|
||||
WorkspaceExecutionStore execution = mock(WorkspaceExecutionStore.class);
|
||||
when(execution.verifiedRecoveryEnabled()).thenReturn(true);
|
||||
when(execution.verifiedRecoveryEnabled()).thenReturn(verifiedRecovery);
|
||||
when(client.capabilities()).thenReturn(capabilities);
|
||||
when(capabilities.getBootId()).thenReturn(BOOT_ID);
|
||||
when(sessions.requireSession("tenant-a", SESSION_ID)).thenReturn(session);
|
||||
@@ -487,7 +488,8 @@ class QwenHostedHarnessConnectorTest {
|
||||
var response = new ObjectMapper().createObjectNode().put("optionId", "allow")
|
||||
.put("inputRevision", 7L).put("policyRevision", "hosted-tool-approval/1");
|
||||
|
||||
doThrow(WorkspaceExecutionStore.unavailable()).doNothing().when(execution).authorize(session);
|
||||
doThrow(WorkspaceExecutionStore.unavailable()).doNothing()
|
||||
.when(execution).authorizeActionResponse(session);
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant-a", SESSION_ID, actionId, response))
|
||||
.hasMessageContaining("Workspace execution authority is unavailable");
|
||||
verify(client, never()).loadSession(any());
|
||||
@@ -504,15 +506,16 @@ class QwenHostedHarnessConnectorTest {
|
||||
for (LoadHarnessSession load : loads.getAllValues()) {
|
||||
Map<String, Object> wire = ReflectionTestUtils.invokeMethod(load, "toJson");
|
||||
assertThat(wire).containsEntry("toolProfile", "hosted-workspace-files/1")
|
||||
.doesNotContainKey("passiveManagedRuntimeRecovery");
|
||||
.containsEntry("passiveManagedRuntimeRecovery", true);
|
||||
assertThat(wire.get("managedSessionStore").toString())
|
||||
.contains("tenantId=tenant-a", "workspaceId=selected-workspace")
|
||||
.doesNotContain("workspaceId=workspace-a");
|
||||
}
|
||||
verify(execution, never()).authorizePassiveAttachment(any());
|
||||
verify(execution, never()).authorize(any());
|
||||
verify(execution, times(2)).verifyMountForProbe(session.workspace());
|
||||
verify(client, never()).createSession(any());
|
||||
|
||||
doThrow(WorkspaceExecutionStore.unavailable()).when(execution).authorize(session);
|
||||
doThrow(WorkspaceExecutionStore.unavailable()).when(execution).authorizeActionResponse(session);
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant-a", SESSION_ID, actionId, response))
|
||||
.hasMessageContaining("Workspace execution authority is unavailable");
|
||||
verify(client, times(1)).resolveAction(any(), any(), any(), anyLong(), any());
|
||||
|
||||
+1
-1
@@ -62,7 +62,7 @@ class WorkspaceMigrationMySqlIT {
|
||||
+ " WHERE installed_rank <= ? ORDER BY installed_rank", lastRank)).isEqualTo(applied);
|
||||
assertThat(jdbc.queryForList("SELECT version FROM flyway_schema_history"
|
||||
+ " WHERE installed_rank > ? AND success = TRUE ORDER BY installed_rank",
|
||||
String.class, lastRank)).containsExactly("48", "49", "50", "51");
|
||||
String.class, lastRank)).containsExactly("48", "49", "50", "51", "52");
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM managed_workspace_migration", Integer.class)).isZero();
|
||||
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM qwen_runtime_storage_fence", Integer.class)).isZero();
|
||||
}
|
||||
|
||||
+172
@@ -2,10 +2,25 @@ package com.alibaba.qwen.code.managedagent.store;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.ArgumentMatchers.anyLong;
|
||||
import static org.mockito.Mockito.doAnswer;
|
||||
import static org.mockito.Mockito.doNothing;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.never;
|
||||
import static org.mockito.Mockito.spy;
|
||||
import static org.mockito.Mockito.verify;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
import com.alibaba.qwen.code.daemon.HarnessSessionRef;
|
||||
import com.alibaba.qwen.code.daemon.HostedHarnessCapabilities;
|
||||
import com.alibaba.qwen.code.daemon.HostedHarnessClient;
|
||||
import com.alibaba.qwen.code.managedagent.config.ManagedAgentProperties;
|
||||
import com.alibaba.qwen.code.managedagent.config.ManagedAgentProperties.RuntimeBroker.WorkspaceMount;
|
||||
import com.alibaba.qwen.code.managedagent.harness.QwenHostedHarnessConnector;
|
||||
import com.alibaba.qwen.code.managedagent.store.StoreModels.SessionRecord;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
import com.alibaba.qwen.code.runtimebroker.JdbcRuntimeBindingRepository;
|
||||
import com.alibaba.qwen.code.runtimebroker.managedworkspace.ContextBinding;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
@@ -29,6 +44,7 @@ import org.junit.jupiter.api.io.TempDir;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
|
||||
import org.springframework.jdbc.datasource.DriverManagerDataSource;
|
||||
import org.springframework.test.util.ReflectionTestUtils;
|
||||
import org.springframework.transaction.PlatformTransactionManager;
|
||||
import org.springframework.transaction.support.DefaultTransactionStatus;
|
||||
|
||||
@@ -233,6 +249,162 @@ class WorkspaceStorageGuardTest {
|
||||
assertUnavailable(() -> unavailable.verify(binding));
|
||||
}
|
||||
|
||||
@Test
|
||||
void actionResponseRetriesAMomentaryMountReadWithoutChangingAcquireVerdicts() {
|
||||
AtomicInteger unreadable = new AtomicInteger();
|
||||
var manager = new DataSourceTransactionManager(dataSource);
|
||||
var guard = new WorkspaceStorageGuard(jdbc, manager, properties, path -> {
|
||||
if (unreadable.get() != 0) {
|
||||
throw new IOException("momentary mount failure");
|
||||
}
|
||||
return new WorkspaceStorageGuard.Identity(path.toString(), "host", "device", "inode",
|
||||
"2026-10-07T00:00:00Z");
|
||||
});
|
||||
guard.register("tenant", "storage", UUID.randomUUID().toString());
|
||||
String sessionId = UUID.randomUUID().toString();
|
||||
var session = new SessionRecord("tenant", sessionId, "qwen-code", null,
|
||||
null, "ACTIVE", null, null, 0, 0, 0, 1, 1, null, 1,
|
||||
binding, "default", "hosted-workspace-files/1");
|
||||
var execution = spy(new WorkspaceExecutionStore(jdbc, manager, guard));
|
||||
doNothing().when(execution).authorizeActionResponse(session);
|
||||
doNothing().when(execution).authorizePassiveAttachment(session);
|
||||
var sessions = mock(AgentStateStore.class);
|
||||
when(sessions.requireSession("tenant", sessionId)).thenReturn(session);
|
||||
var actions = mock(ManagedActionStore.class);
|
||||
when(actions.approvalMode("tenant", sessionId)).thenReturn("default");
|
||||
properties.getHarness().setToken("test-token");
|
||||
properties.getHarness().setCapabilityDigest("sha256:" + "a".repeat(64));
|
||||
properties.getHarness().setWorkspaceFilesEnabled(true);
|
||||
var connector = new QwenHostedHarnessConnector(properties, sessions, execution, actions);
|
||||
var client = mock(HostedHarnessClient.class);
|
||||
var capabilities = mock(HostedHarnessCapabilities.class);
|
||||
when(client.capabilities()).thenReturn(capabilities);
|
||||
when(capabilities.getBootId()).thenReturn(UUID.randomUUID().toString());
|
||||
var attached = mock(HarnessSessionRef.class);
|
||||
when(attached.getApprovalMode()).thenReturn("default");
|
||||
when(client.loadSession(any())).thenReturn(attached);
|
||||
ReflectionTestUtils.setField(connector, "client", client);
|
||||
var response = new ObjectMapper().createObjectNode().put("optionId", "allow")
|
||||
.put("inputRevision", 1).put("policyRevision", "policy");
|
||||
unreadable.set(1);
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant", sessionId, "action", response))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> {
|
||||
assertThat(error.getCode()).isEqualTo("workspace_unavailable");
|
||||
assertThat(error.isRetryable()).isTrue();
|
||||
assertThat(error.getCause()).isInstanceOf(IOException.class);
|
||||
});
|
||||
verify(client, never()).resolveAction(any(), any(), any(), anyLong(), any());
|
||||
assertThatThrownBy(() -> guard.verify(binding))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class,
|
||||
error -> assertThat(error.isRetryable()).isFalse());
|
||||
unreadable.set(0);
|
||||
connector.resolveAction("tenant", sessionId, "action", response);
|
||||
verify(client).resolveAction(attached, "action", "allow", 1L, "policy");
|
||||
// Positive control: the probe-only retry leaves the shared acquire
|
||||
// path's verdict untouched — a settled mount still authorizes.
|
||||
guard.verify(binding);
|
||||
}
|
||||
|
||||
@Test
|
||||
void actionResponseRetriesARevokedGrantWhileKeepingStructuralRefusalsTerminal() {
|
||||
var manager = new DataSourceTransactionManager(dataSource);
|
||||
var guard = new WorkspaceStorageGuard(jdbc, manager, properties, path ->
|
||||
new WorkspaceStorageGuard.Identity(path.toString(), "host", "device", "inode",
|
||||
"2026-10-07T00:00:00Z"));
|
||||
guard.register("tenant", "storage", UUID.randomUUID().toString());
|
||||
// The action-response authority is derived from the real grant rows,
|
||||
// not injected.
|
||||
var execution = spy(new WorkspaceExecutionStore(jdbc, manager, guard));
|
||||
var bound = new ContextBinding("tenant", "workspace", 1, "storage", "child",
|
||||
WorkspaceExecutionProfile.CONTEXT_CONFIG_REF, 1);
|
||||
String sessionId = UUID.randomUUID().toString();
|
||||
jdbc.update("INSERT INTO managed_agent_session (tenant_id, session_id, agent_id, status,"
|
||||
+ " created_at, updated_at, workspace_id, workspace_generation, workspace_storage_id,"
|
||||
+ " cwd_relative, context_config_ref, context_revision, workspace_config_ref,"
|
||||
+ " workspace_policy_ref) VALUES ('tenant', ?, 'qwen-code', 'ACTIVE', 0, 0,"
|
||||
+ " 'workspace', 1, 'storage', 'child', ?, 1, ?, ?)",
|
||||
sessionId, WorkspaceExecutionProfile.CONTEXT_CONFIG_REF,
|
||||
WorkspaceExecutionProfile.CONFIG_REF, WorkspaceExecutionProfile.POLICY_REF);
|
||||
jdbc.update("INSERT INTO managed_workspace_registry (tenant_id, workspace_id,"
|
||||
+ " workspace_generation, storage_id, display_name, config_ref, policy_ref, state)"
|
||||
+ " VALUES ('tenant', 'workspace', 1, 'storage', 'workspace', ?, ?, 'ACTIVE')",
|
||||
WorkspaceExecutionProfile.CONFIG_REF, WorkspaceExecutionProfile.POLICY_REF);
|
||||
jdbc.update("INSERT INTO managed_workspace_create_command (tenant_id, actor_id,"
|
||||
+ " idempotency_key, request_digest, session_id, created_at)"
|
||||
+ " VALUES ('tenant', ?, 'create-1', 'digest', ?, 0)",
|
||||
"owner".getBytes(java.nio.charset.StandardCharsets.UTF_8), sessionId);
|
||||
jdbc.update("INSERT INTO managed_workspace_access (tenant_id, workspace_id, actor_id,"
|
||||
+ " can_read, can_create) VALUES ('tenant', 'workspace', ?, TRUE, TRUE)",
|
||||
"owner".getBytes(java.nio.charset.StandardCharsets.UTF_8));
|
||||
var session = new SessionRecord("tenant", sessionId, "qwen-code", null,
|
||||
null, "ACTIVE", null, null, 0, 0, 0, 1, 1, null, 1,
|
||||
bound, "default", "hosted-workspace-files/1");
|
||||
var sessions = mock(AgentStateStore.class);
|
||||
when(sessions.requireSession("tenant", sessionId)).thenReturn(session);
|
||||
var actions = mock(ManagedActionStore.class);
|
||||
when(actions.approvalMode("tenant", sessionId)).thenReturn("default");
|
||||
properties.getHarness().setToken("test-token");
|
||||
properties.getHarness().setCapabilityDigest("sha256:" + "a".repeat(64));
|
||||
properties.getHarness().setWorkspaceFilesEnabled(true);
|
||||
var connector = new QwenHostedHarnessConnector(properties, sessions, execution, actions);
|
||||
var client = mock(HostedHarnessClient.class);
|
||||
var capabilities = mock(HostedHarnessCapabilities.class);
|
||||
when(client.capabilities()).thenReturn(capabilities);
|
||||
when(capabilities.getBootId()).thenReturn(UUID.randomUUID().toString());
|
||||
var attached = mock(HarnessSessionRef.class);
|
||||
when(attached.getApprovalMode()).thenReturn("default");
|
||||
when(client.loadSession(any())).thenReturn(attached);
|
||||
ReflectionTestUtils.setField(connector, "client", client);
|
||||
var response = new ObjectMapper().createObjectNode().put("optionId", "allow")
|
||||
.put("inputRevision", 1).put("policyRevision", "policy");
|
||||
doAnswer(invocation -> {
|
||||
invocation.callRealMethod();
|
||||
assertThat(jdbc.update("UPDATE managed_workspace_access SET can_create = FALSE"
|
||||
+ " WHERE tenant_id = 'tenant' AND workspace_id = 'workspace'")).isEqualTo(1);
|
||||
return null;
|
||||
}).doCallRealMethod().when(execution).authorizeActionResponse(session);
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant", sessionId, "action", response))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> {
|
||||
assertThat(error.getCode()).isEqualTo("workspace_unavailable");
|
||||
assertThat(error.isRetryable()).isTrue();
|
||||
});
|
||||
// A revoked creation grant is operator-reversible: the refusal must
|
||||
// stay retryable so a restored grant still delivers the answer.
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = FALSE"
|
||||
+ " WHERE tenant_id = 'tenant' AND workspace_id = 'workspace'");
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant", sessionId, "action", response))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> {
|
||||
assertThat(error.getCode()).isEqualTo("workspace_unavailable");
|
||||
assertThat(error.isRetryable()).isTrue();
|
||||
});
|
||||
// So is a registry that left ACTIVE.
|
||||
jdbc.update("UPDATE managed_workspace_access SET can_create = TRUE"
|
||||
+ " WHERE tenant_id = 'tenant' AND workspace_id = 'workspace'");
|
||||
jdbc.update("UPDATE managed_workspace_registry SET state = 'DRAINING'"
|
||||
+ " WHERE tenant_id = 'tenant' AND workspace_id = 'workspace'");
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant", sessionId, "action", response))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> {
|
||||
assertThat(error.getCode()).isEqualTo("workspace_unavailable");
|
||||
assertThat(error.isRetryable()).isTrue();
|
||||
});
|
||||
verify(client, never()).resolveAction(any(), any(), any(), anyLong(), any());
|
||||
// Restored, the committed decision reaches the Harness.
|
||||
jdbc.update("UPDATE managed_workspace_registry SET state = 'ACTIVE'"
|
||||
+ " WHERE tenant_id = 'tenant' AND workspace_id = 'workspace'");
|
||||
connector.resolveAction("tenant", sessionId, "action", response);
|
||||
verify(client).resolveAction(attached, "action", "allow", 1L, "policy");
|
||||
// Generation drift after a re-registration stays structural: the
|
||||
// terminal exit keeps its verdict.
|
||||
jdbc.update("UPDATE managed_workspace_registry SET workspace_generation = 2"
|
||||
+ " WHERE tenant_id = 'tenant' AND workspace_id = 'workspace'");
|
||||
assertThatThrownBy(() -> connector.resolveAction("tenant", sessionId, "action", response))
|
||||
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> {
|
||||
assertThat(error.getCode()).isEqualTo("workspace_unavailable");
|
||||
assertThat(error.isRetryable()).isFalse();
|
||||
});
|
||||
verify(client).resolveAction(any(), any(), any(), anyLong(), any());
|
||||
}
|
||||
|
||||
@Test
|
||||
void refusesDirectoryAndSymlinkMarkersWithoutChangingRegistration() throws Exception {
|
||||
String operation = UUID.randomUUID().toString();
|
||||
|
||||
@@ -901,6 +901,34 @@ describe('ManagedSessionsPage', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each([true, false])(
|
||||
'lets the creator cancel a running bound Turn when workspaceTurns is %s',
|
||||
async (workspaceTurns) => {
|
||||
mocks.client.getSession.mockResolvedValue(
|
||||
summary('bound', {
|
||||
phase: 'tool_running',
|
||||
workspace: { workspaceId: 'ws-a', cwdRelative: 'services/api' },
|
||||
capabilities: {
|
||||
canSend: false,
|
||||
canCancel: true,
|
||||
...(workspaceTurns ? { workspaceTurns: true } : {}),
|
||||
},
|
||||
}),
|
||||
);
|
||||
await render('bound');
|
||||
|
||||
// Without workspaceTurns the composer stays hidden, but Cancel does not.
|
||||
expect(container.querySelector('textarea') !== null).toBe(workspaceTurns);
|
||||
await click('Cancel turn');
|
||||
|
||||
expect(mocks.client.cancel).toHaveBeenCalledWith(
|
||||
'bound',
|
||||
'p1',
|
||||
expect.objectContaining({ clientId: expect.any(String) }),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
async function click(label: string) {
|
||||
const button = [...container.querySelectorAll('button')].find(
|
||||
(item) => item.textContent === label,
|
||||
|
||||
@@ -405,6 +405,22 @@ function ManagedSessionsContent({
|
||||
const active =
|
||||
summary &&
|
||||
!['created', 'completed', 'failed', 'cancelled'].includes(summary.phase);
|
||||
// Cancel authority differs from submit authority: the creator may stop a
|
||||
// running bound Turn after the Workspace stops admitting new work, so the
|
||||
// control is not tied to the composer and the server's 409 is the gate.
|
||||
const cancelButton =
|
||||
cancellationEnabled &&
|
||||
summary?.capabilities.canCancel &&
|
||||
summary.activeTurnId ? (
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
disabled={busy}
|
||||
onClick={() => void cancel()}
|
||||
>
|
||||
{t('managed.cancel')}
|
||||
</Button>
|
||||
) : null;
|
||||
return (
|
||||
<div className="flex h-full min-h-0 flex-col gap-3">
|
||||
{/* The uncertain-request row unmounts on the Discard click itself, so
|
||||
@@ -732,18 +748,7 @@ function ManagedSessionsContent({
|
||||
{t('managed.newRequired')}
|
||||
</span>
|
||||
)}
|
||||
{cancellationEnabled &&
|
||||
summary?.capabilities.canCancel &&
|
||||
summary.activeTurnId && (
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
disabled={busy}
|
||||
onClick={() => void cancel()}
|
||||
>
|
||||
{t('managed.cancel')}
|
||||
</Button>
|
||||
)}
|
||||
{cancelButton}
|
||||
<Button
|
||||
type="submit"
|
||||
disabled={
|
||||
@@ -761,7 +766,11 @@ function ManagedSessionsContent({
|
||||
</Button>
|
||||
</div>
|
||||
</form>
|
||||
) : null}
|
||||
) : (
|
||||
cancelButton && (
|
||||
<div className="flex shrink-0 justify-end">{cancelButton}</div>
|
||||
)
|
||||
)}
|
||||
{outputTarget &&
|
||||
outputTarget.sessionId === sessionId &&
|
||||
sessionId &&
|
||||
|
||||
@@ -76,7 +76,7 @@ export interface paths {
|
||||
};
|
||||
get?: never;
|
||||
put?: never;
|
||||
/** @description Maps workspaceId/cwdRelative to public Workspace selection without using environmentId or absolute cwd. Shares G0's opt-in initial file-tool Turn admission and fixed server-owned profile with public Session creation. The Session creator may submit later Turns and cancel its running Turns under the same opt-in, and may rename the Session, while the creator currently holds Workspace read and create grants on a registry row whose state is ACTIVE and the Session is an active, undeleted qwen-code Session on the frozen execution profile; Workspace close follows its separate close capability and lifecycle admission; archive and unarchive follow retention capabilities after reliable Workspace close; delete supports either L3 idle ACTIVE files Sessions or L2 CLOSED/ARCHIVED Sessions; controlled same-Workspace cwd changes ship through the durable cwd_change operations (v1.32). The per-caller workspaceTurns capability on this surface advertises the same rule. Freeze selection with the original idempotency key; admission does not prove physical directory readiness. */
|
||||
/** @description Maps workspaceId/cwdRelative to public Workspace selection without using environmentId or absolute cwd. Shares G0's opt-in initial file-tool Turn admission and fixed server-owned profile with public Session creation. The Session creator may submit later Turns under the same opt-in with current read and create grants on the bound ACTIVE Workspace generation and storage. Cancellation of running work requires the creator’s read grant and the opt-in, even after create authority is revoked (a live cancel reuses the owner's resident Harness attachment; a cold connector cache passively re-attaches for a persisted cancellation after frozen identity checks, independently of current creation grants, registry state or mount readiness; new work still rechecks execution authority); Workspace close follows its separate close capability and lifecycle admission; archive and unarchive follow retention capabilities after reliable Workspace close; delete supports either L3 idle ACTIVE files Sessions or L2 CLOSED/ARCHIVED Sessions; controlled same-Workspace cwd changes ship through the durable cwd_change operations (v1.32). The per-caller workspaceTurns capability on this surface advertises current submit and rename admission; it does not gate cancellation. Freeze selection with the original idempotency key; admission does not prove physical directory readiness. */
|
||||
post: operations["webShellCreateSession"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
@@ -530,10 +530,10 @@ export interface components {
|
||||
/** @default false */
|
||||
actions: boolean;
|
||||
/**
|
||||
* @description True when the caller may submit later Turns of this Workspace-bound Session, cancel its running Turns and rename it: the deployment enables Workspace files, the caller created the Session and currently holds Workspace read and create grants on a registry row whose state is ACTIVE, and the Session is an active qwen-code Session on the frozen execution profile. False for every other caller and for unbound Sessions, which do not use it.
|
||||
* @description True when the caller may submit later Turns of this Workspace-bound Session: the deployment enables Workspace files, the caller created the Session and currently holds Workspace read and create grants on a registry row whose state is ACTIVE and still carries the Workspace generation and storage identity the Session was bound to, and the Session is an active, undeleted qwen-code Session on the frozen execution profile. Cancelling only aborts work that is already running, so the creator may cancel a running Turn even when this is false, as long as they can still read the Workspace and the deployment still enables Workspace files. False for every other caller and for unbound Sessions, which do not use it.
|
||||
* @default false
|
||||
*/
|
||||
workspaceTurns?: boolean;
|
||||
workspaceTurns: boolean;
|
||||
tasks: boolean;
|
||||
artifacts: boolean;
|
||||
/**
|
||||
|
||||
@@ -893,31 +893,41 @@ describe('createJavaManagedAgentProvider', () => {
|
||||
).toEqual({ canSend: false, canCancel: false });
|
||||
});
|
||||
|
||||
it('lets the allowed caller cancel a running Turn of a bound Session', async () => {
|
||||
const provider = createJavaManagedAgentProvider({
|
||||
baseUrl: 'https://product.example',
|
||||
fetch: vi.fn<typeof fetch>().mockResolvedValue(
|
||||
jsonResponse({
|
||||
sessionId: 'bound-1',
|
||||
status: 'ACTIVE',
|
||||
createdAt: 1,
|
||||
updatedAt: 2,
|
||||
lastSequence: 5,
|
||||
workspace: { workspaceId: 'ws-a', cwdRelative: '.' },
|
||||
activeTurn: {
|
||||
turnId: 'turn-2',
|
||||
it.each([true, false])(
|
||||
'offers cancel for a running bound Turn when workspaceTurns is %s',
|
||||
async (workspaceTurns) => {
|
||||
const provider = createJavaManagedAgentProvider({
|
||||
baseUrl: 'https://product.example',
|
||||
fetch: vi.fn<typeof fetch>().mockResolvedValue(
|
||||
jsonResponse({
|
||||
sessionId: 'bound-1',
|
||||
status: 'RUNNING',
|
||||
submittedAt: 2,
|
||||
},
|
||||
capabilities: { tasks: true, workspaceTurns: true },
|
||||
}),
|
||||
),
|
||||
});
|
||||
expect(
|
||||
(await provider.getSession('bound-1', { clientId: 'c' })).capabilities,
|
||||
).toEqual({ canSend: false, canCancel: true, workspaceTurns: true });
|
||||
});
|
||||
status: 'ACTIVE',
|
||||
createdAt: 1,
|
||||
updatedAt: 2,
|
||||
lastSequence: 5,
|
||||
workspace: { workspaceId: 'ws-a', cwdRelative: '.' },
|
||||
activeTurn: {
|
||||
turnId: 'turn-2',
|
||||
sessionId: 'bound-1',
|
||||
status: 'RUNNING',
|
||||
submittedAt: 2,
|
||||
},
|
||||
capabilities: { tasks: true, workspaceTurns },
|
||||
}),
|
||||
),
|
||||
});
|
||||
// The creator may still cancel after the Workspace stops admitting
|
||||
// new work, so cancel does not follow workspaceTurns; the server
|
||||
// refuses anyone else.
|
||||
expect(
|
||||
(await provider.getSession('bound-1', { clientId: 'c' })).capabilities,
|
||||
).toEqual({
|
||||
canSend: false,
|
||||
canCancel: true,
|
||||
...(workspaceTurns ? { workspaceTurns: true } : {}),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it('passes download cancellation through the host sink to the content fetch', async () => {
|
||||
const abort = new AbortController();
|
||||
|
||||
@@ -353,11 +353,10 @@ function toSessionSummary(
|
||||
...(session.capabilities?.artifacts === true ? { artifacts: true } : {}),
|
||||
canSend:
|
||||
sessionActive && !active && (!session.workspace || workspaceTurns),
|
||||
canCancel:
|
||||
sessionActive &&
|
||||
active &&
|
||||
turnStatus !== 'cancelling' &&
|
||||
(!session.workspace || workspaceTurns),
|
||||
// The creator may cancel a running bound Turn after the Workspace stops
|
||||
// admitting new work, so cancel is not tied to workspaceTurns; the
|
||||
// server refuses anyone else.
|
||||
canCancel: sessionActive && active && turnStatus !== 'cancelling',
|
||||
...(workspaceTurns ? { workspaceTurns: true } : {}),
|
||||
...(session.capabilities?.actions === true ? { actions: true } : {}),
|
||||
},
|
||||
|
||||
@@ -58,7 +58,12 @@ export function javaSessionPayload(lastSequence: number): string {
|
||||
createdAt: 1,
|
||||
updatedAt: 1,
|
||||
lastSequence,
|
||||
capabilities: { tasks: true, artifacts: false, actions: false },
|
||||
capabilities: {
|
||||
tasks: true,
|
||||
artifacts: false,
|
||||
actions: false,
|
||||
workspaceTurns: false,
|
||||
},
|
||||
};
|
||||
return JSON.stringify(payload);
|
||||
}
|
||||
@@ -39,7 +39,12 @@ async function installManagedScenario(page: Page, testInfo: TestInfo) {
|
||||
},
|
||||
environment: { state: 'ready' },
|
||||
lastSequence: 0,
|
||||
capabilities: { tasks: true, artifacts: false, actions: false },
|
||||
capabilities: {
|
||||
tasks: true,
|
||||
artifacts: false,
|
||||
actions: false,
|
||||
workspaceTurns: false,
|
||||
},
|
||||
};
|
||||
function append(type: string, data: Record<string, unknown> = {}) {
|
||||
const sequence = events.length + 1;
|
||||
|
||||
Reference in new issue
Block a user