mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
test(managed-agent): gate lifecycle-v2 close with a stopped worker (#16123)
Co-authored-by: Qwen-Coder <[email protected]>
This commit is contained in:
Shaojin WenandQwen-Coder authored and GitHub committed
1 parent
b87d60ddf0
commit
edb094fea3
6 files changed
+194
-12
No files matched your search
@@ -1139,7 +1139,7 @@ jobs:
|
||||
name: 'Hosted process fault gates / MySQL 8.4 / Java 21'
|
||||
if: "${{ inputs.tier != 'fast' }}"
|
||||
runs-on: 'ubuntu-latest'
|
||||
# The step ceilings today (204 + 20 + 50 + 20 = 294 — the FG8 letters
|
||||
# The step ceilings today (205 + 20 + 50 + 20 = 295 — the FG8 letters
|
||||
# grew the shared failsafe fork to 3600 s, FG9 slice 3 to 3990 s, the
|
||||
# FG10b–f workflow relay letters to 4484 s, the FG12a MCP reply-loss
|
||||
# letter to 4499 s, the FG12c restart matrix to 4548 s, the FG13c Hook
|
||||
@@ -1219,18 +1219,21 @@ jobs:
|
||||
# of fork plus wrap, 44 s left); the FG12d detach-unknown cell adds 55 s
|
||||
# (27.4 s measured), so the fork moves to 11591 s and Verify to 204
|
||||
# minutes, wrapping fork + 600 s plus margin (12191 s of fork plus wrap,
|
||||
# 49 s left);
|
||||
# 49 s left); the FG12d stopped-worker close cell adds 67 s (33.1 s
|
||||
# measured), so the fork moves to 11658 s and Verify to 205 minutes,
|
||||
# wrapping fork + 600 s plus margin (12258 s of fork plus wrap, 42 s
|
||||
# left);
|
||||
# the Broker fault gates use 20,
|
||||
# and admission plus O4 keep per-fork ceilings) plus a 33-minute
|
||||
# allowance for the uncapped setup steps (Checkout.. cgroup delegation,
|
||||
# the report checks), which took under 10 minutes on every nightly from
|
||||
# 10-14 to 10-30: 294 + 33. A Verify that timed out used to skip
|
||||
# 10-14 to 10-30: 295 + 33. A Verify that timed out used to skip
|
||||
# admission, leaving that path 50 minutes spare; admission now runs
|
||||
# after it, so that path too can fill all four capped ceilings. The
|
||||
# allowance sits above the ~13-minute floor so that a slow setup (the
|
||||
# CLI install makes up to three attempts) still leaves the O4 gates
|
||||
# time to run. The failover arms run in hosted-failover-e2e.
|
||||
timeout-minutes: 327
|
||||
timeout-minutes: 328
|
||||
services:
|
||||
mysql:
|
||||
image: 'mysql:8.4.6'
|
||||
@@ -1330,10 +1333,10 @@ jobs:
|
||||
# fork leaves no per-test failure lines, so the main-CI failure
|
||||
# analyzer can only file an undiagnosable per-commit issue (#13503).
|
||||
# hosted-process-ci.test.js pins this ordering; the fork budget is
|
||||
# 11591 s, so the step wraps to fork + 600 s plus margin (204 min,
|
||||
# 49 s left before the step must move), including the provisional
|
||||
# 11658 s, so the step wraps to fork + 600 s plus margin (205 min,
|
||||
# 42 s left before the step must move), including the provisional
|
||||
# 180 s pool-fixture reservation pending CI measurement.
|
||||
timeout-minutes: 204
|
||||
timeout-minutes: 205
|
||||
env:
|
||||
MAVEN_ARGS: '--settings ${{ runner.temp }}/setup-java-m2/settings.xml --toolchains ${{ runner.temp }}/setup-java-m2/toolchains.xml'
|
||||
MYSQL_PORT: "${{ job.services.mysql.ports['3306'] }}"
|
||||
|
||||
@@ -150,3 +150,22 @@ Decisions:
|
||||
Mutation witness: wrap `session.mcp?.close()` in the close path of a packaged CLI copy in `try { … } catch {}`; detach answers 204 and the case is red at the status assertion. The restored bundle is green.
|
||||
|
||||
Budget: the case measures 27.4 s worst of two runs (27.4 s and 17.4 s) on JDK 21, MySQL 8.4.6 and Linux x86_64; booked at 55 s (2×). The `hosted-harness-mysql` fork moves to 11591 s, the Verify step to 204 minutes and the job ceiling to 327 (294 + 33).
|
||||
|
||||
## Follow-up cell: lifecycle-v2 close with a stopped worker
|
||||
|
||||
#15750's FG12d row also asks for a lifecycle-v2 close that meets a **stopped worker**, and its stop proof. The R1-1 cell replaces the Broker while the worker lives; this follow-up keeps the Broker and kills the durable worker after the configuration has drained, so the close can only finish through the original worker's persisted stop receipt.
|
||||
|
||||
| Case | Injection | Expect |
|
||||
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `close-stopped-worker-stop-proof` | Drive the v2 close to `prepare_resources`; refuse the first owner release before forwarding so the configuration commits `drained` while the worker is alive; SIGKILL the durable worker; the next settle refuses; run the coordinator's MCP recovery arm (`recoverStoppedWorkspaceResources`) under the same claim; retry the settle | The first settle answers 503 `workspace_lifecycle_resources_unsettled` with the record `drained`, one `mcp-release` and one `mcp.exit`. The post-kill settle answers 503 as well. The recovery retires the binding: `binding_state` `RELEASED` with a persisted `drain_receipt_json` whose `bindingId` and `generation` match the retired row, the Runtime Session `RELEASED` and the storage holder cleared. The retried settle answers 200 and the configuration is `released`. Every proxy call after the kill is an owner release (no acquire, no MCP control); exactly one `mcp.start`, no replacement worker. |
|
||||
|
||||
Decisions:
|
||||
|
||||
1. **The worker dies after `drained`, not before.** A worker killed with the configuration still `active` leaves an undispatchable `mcp-release` — the unknown/blocked form the detach-unknown cell already gates. The stop-proof path is the release's: the configuration drains on the live worker, then the close must retire the binding without it.
|
||||
2. **The stop proof is the persisted drain receipt.** The recovery arm observes the original worker absent with matching loss evidence, stops it durably, and persists the receipt **before** it releases the session rows (`releaseStoppedSession` refuses a null receipt) and clears the storage holders (`releaseRetiredHolders` only clears on a persisted receipt). The cell reads `ma_runtime_binding.drain_receipt_json` and matches its `bindingId`/`generation` against the retired row; `RELEASED` without a receipt would be a fabricated retirement.
|
||||
3. **The test plays the coordinator's recovery trigger.** `SessionLifecycleCoordinator` runs `recoverStoppedWorkspaceResources` when the Harness settle fails for the MCP profile; the cell calls the same embedded-Broker arm under the admitted close's `{operationId, claimGeneration}`, exactly as it plays the DRAINING fence with a SQL update. The coordinator's trigger logic is not the subject — the Broker's drain and stop-proof machinery is. This extends the slice's "drive lifecycle v2 directly" decision to the one phase the Harness cannot finish alone.
|
||||
4. **No exchange with the dead process.** After the kill the proxy sees only owner releases: the blocked settle's 503 and the retried settle's 200. The recovery is in-JVM. No acquire and no MCP control may cross, and no second `mcp.start` may appear.
|
||||
|
||||
Mutation witness: skip `renewal.persistDrainReceipt(proof, authority)` in `drainClaimedBinding`'s absence arm. The recovery then fails at the stop-proof step (`Original drain claim expired`) and the case errors before the binding assertions; the restored file is byte-identical (sha256-verified) and the case is green again.
|
||||
|
||||
Budget: the case measures 33.09 s and 27.16 s standalone (18.21 s on the post-mutation restore run) on JDK 21, MySQL 8.4.6 and Linux x86_64; the twelve-case class is 152.9 s. It adds 67 s (2× the 33.1 s worst) to the shared healthy sum (11591 → 11658 s), so the `hosted-harness-mysql` fork moves to 11658 s, the Verify step to 205 minutes (12258 s of fork plus wrap, 42 s left) and the job ceiling to 328 (295 + 33).
|
||||
@@ -150,3 +150,22 @@ v2 关闭被准入之后，store 会 fence 普通写者续租，而 Harness 要
|
||||
变异见证：在打包 CLI 副本的 close 路径中把 `session.mcp?.close()` 包进 `try { … } catch {}`；detach 返回 204，用例在状态断言处变红。还原后的 bundle 复绿。
|
||||
|
||||
预算：该用例在 JDK 21、MySQL 8.4.6、Linux x86_64 上两次实测取最慢 27.4 秒（27.4/17.4），按 2× 记 55 秒。`hosted-harness-mysql` fork 移到 11591 秒，Verify 移到 204 分钟，job 上限移到 327（294 + 33）。
|
||||
|
||||
## 后续格：stopped worker 的 lifecycle-v2 close
|
||||
|
||||
#15750 的 FG12d 行还要求一格遇到**已停 worker** 的 lifecycle-v2 close，及其 stop proof。R1-1 格在 worker 存活时替换 Broker；本格保留 Broker，在配置已 drained 之后杀掉 durable worker，因此 close 只能凭原 worker 持久化的 stop receipt 完成。
|
||||
|
||||
| 用例 | 注入 | 期望 |
|
||||
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `close-stopped-worker-stop-proof` | 把 v2 close 驱动到 `prepare_resources`；在转发前拒绝第一次 owner release，让配置在 worker 存活时提交 `drained`；SIGKILL durable worker；下一次 settle 被拒；在同一 claim 下执行协调器的 MCP recovery 臂（`recoverStoppedWorkspaceResources`）；重试 settle | 第一次 settle 答 503 `workspace_lifecycle_resources_unsettled`，记录 `drained`、一个 `mcp-release`、一个 `mcp.exit`。kill 后的 settle 同样答 503。recovery 退休 binding：`binding_state` `RELEASED` 且持久化了 `drain_receipt_json`，其 `bindingId`/`generation` 与退休行一致，Runtime Session `RELEASED`、storage holder 清空。重试的 settle 答 200，配置 `released`。kill 之后每个代理调用都是 owner release（无 acquire、无 MCP 控制）；恰好一个 `mcp.start`，没有替换 worker。 |
|
||||
|
||||
决策：
|
||||
|
||||
1. **worker 死在 `drained` 之后，而非之前。** 配置仍 `active` 时杀 worker，留下的是无法派发的 `mcp-release`——那是 detach-unknown 格已门禁的未知/受阻形态。stop-proof 路径属于 release：配置在活 worker 上 drain，然后 close 必须在没有 worker 的情况下退休 binding。
|
||||
2. **stop proof 就是持久化的 drain receipt。** recovery 臂观察到原 worker 缺失且 loss evidence 匹配，先 durable stop、持久化 receipt，**然后**才释放 session 行（`releaseStoppedSession` 拒绝空 receipt）、清空 storage holder（`releaseRetiredHolders` 只凭持久化 receipt 清空）。用例读取 `ma_runtime_binding.drain_receipt_json`，把它的 `bindingId`/`generation` 与退休行比对；没有 receipt 的 `RELEASED` 就是伪造的退休。
|
||||
3. **测试代打协调器的 recovery 触发。** `SessionLifecycleCoordinator` 在 MCP profile 的 Harness settle 失败时执行 `recoverStoppedWorkspaceResources`；本格以已准入 close 的 `{operationId, claimGeneration}` 调用同一个 embedded-Broker 臂，正如它用 SQL 更新代打 DRAINING 围栏。协调器的触发逻辑不是被测对象——Broker 的 drain 与 stop-proof 机制才是。这把本切片「直接驱动 lifecycle v2」的决策扩展到 Harness 独自无法完成的这一个阶段。
|
||||
4. **不与死进程交换。** kill 之后代理只看到 owner release：受阻 settle 的 503 与重试 settle 的 200。recovery 在 JVM 内。不得有 acquire 或 MCP 控制越过，也不得出现第二个 `mcp.start`。
|
||||
|
||||
变异见证：在 `drainClaimedBinding` 的 absence 臂跳过 `renewal.persistDrainReceipt(proof, authority)`。recovery 随即在 stop-proof 步骤失败（`Original drain claim expired`），用例在 binding 断言之前报错；还原后的文件逐字节一致（sha256 校验），用例复绿。
|
||||
|
||||
预算：用例在 JDK 21、MySQL 8.4.6、Linux x86_64 上单独测得 33.09 s 与 27.16 s（变异还原后的运行 18.21 s）；十二用例整类 152.9 s。按最坏 33.1 s 的 2× 计 67 s 加入共享健康总和（11591 → 11658 s），`hosted-harness-mysql` fork 移至 11658 s，Verify 步骤移至 205 分钟（fork 加 wrap 共 12258 s，余 42 s），job 上限移至 328（295 + 33）。
|
||||
@@ -319,7 +319,7 @@
|
||||
qwen.hosted.mysql.isolation = true
|
||||
</configurationParameters>
|
||||
</properties>
|
||||
<forkedProcessTimeoutInSeconds>11591</forkedProcessTimeoutInSeconds>
|
||||
<forkedProcessTimeoutInSeconds>11658</forkedProcessTimeoutInSeconds>
|
||||
</configuration>
|
||||
<executions>
|
||||
<execution>
|
||||
|
||||
+139
@@ -1,10 +1,12 @@
|
||||
package com.alibaba.qwen.code.managedagent;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.config.runtime.EmbeddedRuntimeBroker;
|
||||
import com.alibaba.qwen.code.managedagent.contracts.session.SessionCreationCommand;
|
||||
import com.alibaba.qwen.code.managedagent.contracts.workspace.WorkspaceSelection;
|
||||
import com.alibaba.qwen.code.managedagent.model.StoreModels.OperationRecord;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.ManagedAgentStore;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceStorageGuard;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeLifecycleAuthority;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
import com.alibaba.qwen.code.runtimebroker.managedschema.ManagedSchemaTime;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
@@ -68,6 +70,9 @@ import static org.awaitility.Awaitility.await;
|
||||
// the first configuration's same-id retry to release it before the second.
|
||||
// Its R1-1 cell refuses the owner release after `drained` and replaces
|
||||
// Spring, and with it the embedded Broker, before the retry.
|
||||
// Its stopped-worker cell kills the durable worker after `drained` and
|
||||
// requires the close to retire the binding through the coordinator's
|
||||
// stop-proof recovery and the persisted drain receipt.
|
||||
class HostedMcpRestartFaultGatesIT {
|
||||
private static final String PROFILE = "hosted-workspace-mcp/1";
|
||||
// Per-case evidence letter: FG12c keeps its directory, FG12d's release
|
||||
@@ -94,6 +99,8 @@ class HostedMcpRestartFaultGatesIT {
|
||||
Map.entry("cancelBeforeDispatchIsNotStartedProvenOnMySql",
|
||||
"fg12d"),
|
||||
Map.entry("ownerReleaseRefusedThenSpringRestartReleasesTheHolderOnMySql",
|
||||
"fg12d"),
|
||||
Map.entry("closeWithStoppedWorkerReleasesThroughTheStopProofOnMySql",
|
||||
"fg12d"));
|
||||
private static final String TOKEN = "extension-process-fixture-token";
|
||||
private static final String CAPABILITY = "sha256:" + "a".repeat(64);
|
||||
@@ -1245,6 +1252,138 @@ class HostedMcpRestartFaultGatesIT {
|
||||
assertThat(fixtureEvents("mcp.exit")).hasSize(1);
|
||||
}
|
||||
|
||||
// FG12d (issue #15750): lifecycle-v2 close meets a stopped worker. The
|
||||
// configuration drains while the worker is alive; the worker then dies
|
||||
// before the owner release. The Harness settle refuses, the coordinator's
|
||||
// MCP recovery arm retires the binding through the original worker's
|
||||
// stop proof — the drain receipt persisted before its rows are released
|
||||
// — and the retried settle completes. No exchange with the dead process
|
||||
// and no replacement worker.
|
||||
@Test
|
||||
@Timeout(600)
|
||||
void closeWithStoppedWorkerReleasesThroughTheStopProofOnMySql()
|
||||
throws Exception {
|
||||
SessionRef session = createSession();
|
||||
baseline(session);
|
||||
jdbc.update("UPDATE ma_session SET tool_profile = ? WHERE tenant_id ="
|
||||
+ " ? AND session_id = ?", PROFILE, tenant, session.id());
|
||||
OperationRecord close = admitClose(session);
|
||||
String closeId = close.operationId();
|
||||
long claimGeneration = close.claimGeneration();
|
||||
driveLifecycle(session, closeId, claimGeneration, "effects");
|
||||
jdbc.update("UPDATE ma_session_head SET phase = 'DRAINING' WHERE"
|
||||
+ " tenant_id = ? AND harness_session_id = ? AND"
|
||||
+ " operation_id = ?", tenant, session.id(), closeId);
|
||||
driveLifecycle(session, closeId, claimGeneration,
|
||||
"prepare_resources");
|
||||
|
||||
refuseNextOwnerRelease();
|
||||
HttpResponse<String> refused = rawLifecycle(session, closeId,
|
||||
claimGeneration, "settle_resources");
|
||||
assertThat(refused.statusCode()).as(refused.body()).isEqualTo(503);
|
||||
assertThat(json.readTree(refused.body()).path("code").asText())
|
||||
.isEqualTo("workspace_lifecycle_resources_unsettled");
|
||||
assertThat(configuration(session.id()).path("releaseState").asText())
|
||||
.as("the configuration drains while the worker is alive")
|
||||
.isEqualTo("drained");
|
||||
assertThat(controls("mcp-release")).hasSize(1);
|
||||
assertThat(fixtureEvents("mcp.exit")).hasSize(1);
|
||||
assertThat(runtimeState(session)).extracting(
|
||||
row -> row.get("session_state")).containsExactly("READY");
|
||||
|
||||
long worker = awaitWorkerPid();
|
||||
kill(worker);
|
||||
assertThat(ProcessHandle.of(worker)
|
||||
.map(ProcessHandle::isAlive).orElse(false)).isFalse();
|
||||
int killSeq = proxySequence.get();
|
||||
|
||||
// The Harness settle cannot release a dead worker's owner: the
|
||||
// in-memory join is unusable and the saved-session arm needs the
|
||||
// coordinator's stop-proof recovery first.
|
||||
HttpResponse<String> blocked = rawLifecycle(session, closeId,
|
||||
claimGeneration, "settle_resources");
|
||||
assertThat(blocked.statusCode()).as(blocked.body()).isEqualTo(503);
|
||||
assertThat(json.readTree(blocked.body()).path("code").asText())
|
||||
.as("settle: %s", blocked.body())
|
||||
.isIn("workspace_lifecycle_resources_unsettled",
|
||||
"workspace_close_identity_unverified");
|
||||
|
||||
// The coordinator's MCP recovery arm (SessionLifecycleCoordinator's
|
||||
// recoverStoppedWorkspaceResources on a HarnessFailure): drain the
|
||||
// stopped binding under the same claim. The stop receipt must be
|
||||
// persisted before the session rows are released.
|
||||
spring.getBean(EmbeddedRuntimeBroker.class)
|
||||
.recoverStoppedWorkspaceResources(tenant, session.id(),
|
||||
new RuntimeLifecycleAuthority(closeId,
|
||||
claimGeneration))
|
||||
.toCompletableFuture().get(60, TimeUnit.SECONDS);
|
||||
Map<String, Object> binding = jdbc.queryForMap("SELECT"
|
||||
+ " CAST(b.binding_id AS CHAR) AS binding_id,"
|
||||
+ " b.binding_state, b.runtime_generation,"
|
||||
+ " b.drain_receipt_json FROM ma_runtime_session s JOIN"
|
||||
+ " ma_runtime_binding b ON s.binding_id = b.binding_id"
|
||||
+ " WHERE s.tenant_id = ? AND s.harness_session_id = ?",
|
||||
tenant, session.id());
|
||||
assertThat(binding.get("binding_state"))
|
||||
.as("the binding retires on the stop proof, not on loss")
|
||||
.isEqualTo("RELEASED");
|
||||
assertThat(binding.get("drain_receipt_json"))
|
||||
.as("the original worker's stop receipt is persisted")
|
||||
.isNotNull();
|
||||
JsonNode receipt = json.readTree(
|
||||
(String) binding.get("drain_receipt_json"));
|
||||
assertThat(receipt.path("bindingId").asText())
|
||||
.isEqualTo(binding.get("binding_id"));
|
||||
assertThat(receipt.path("generation").asLong())
|
||||
.isEqualTo(((Number) binding.get("runtime_generation"))
|
||||
.longValue());
|
||||
assertThat(receipt.path("stoppedAt").asText()).isNotBlank();
|
||||
assertThat(runtimeState(session)).extracting(
|
||||
row -> row.get("session_state")).containsExactly("RELEASED");
|
||||
assertThat(storageHolders(session))
|
||||
.as("the stop proof releases the storage holder")
|
||||
.isZero();
|
||||
|
||||
HttpResponse<String> settled = null;
|
||||
for (int attempt = 0; attempt < 30 && settled == null; attempt++) {
|
||||
HttpResponse<String> response = rawLifecycle(session, closeId,
|
||||
claimGeneration, "settle_resources");
|
||||
if (response.statusCode() == 200) {
|
||||
settled = response;
|
||||
break;
|
||||
}
|
||||
String code = json.readTree(response.body()).path("code")
|
||||
.asText();
|
||||
assertThat(code).as("settle: %s", response.body())
|
||||
.isIn("workspace_lifecycle_resources_unsettled",
|
||||
"hosted_turn_active");
|
||||
Thread.sleep(1_000);
|
||||
}
|
||||
assertThat(settled).as("the retried settle completes the close")
|
||||
.isNotNull();
|
||||
|
||||
assertThat(proxyCalls.stream()
|
||||
.filter(call -> call.seq() > killSeq).toList())
|
||||
.as("after the worker dies the close only resends the owner"
|
||||
+ " release: no acquire and no MCP control")
|
||||
.isNotEmpty()
|
||||
.allSatisfy(call -> {
|
||||
assertThat(call.path()).endsWith(":release");
|
||||
assertThat(call.kind()).isNull();
|
||||
});
|
||||
assertThat(ownerReleaseCalls()).extracting(ProxyCall::status)
|
||||
.startsWith(503)
|
||||
.endsWith(200);
|
||||
assertThat(configuration(session.id()).path("releaseState").asText())
|
||||
.isEqualTo("released");
|
||||
assertThat(fixtureEvents("mcp.start"))
|
||||
.as("no replacement worker starts a second server")
|
||||
.hasSize(1);
|
||||
assertThat(controls("mcp-release"))
|
||||
.as("the only release dispatch happened before the kill")
|
||||
.hasSize(1);
|
||||
}
|
||||
|
||||
private void refuseNextOwnerRelease() {
|
||||
ownerReleaseRefuseArmed.set(true);
|
||||
}
|
||||
|
||||
@@ -661,13 +661,13 @@ describe('Hosted real-process gates', () => {
|
||||
});
|
||||
|
||||
it.each([
|
||||
// Step ceilings today (204 + 20 + 50 + 20): Verify, the Broker fault
|
||||
// Step ceilings today (205 + 20 + 50 + 20): Verify, the Broker fault
|
||||
// gates, admission and O4. The failover arms moved to their own job.
|
||||
// The MySQL job's ceiling carries 33 minutes of setup allowance, above
|
||||
// the 13-minute floor: a Verify that timed out used to skip admission,
|
||||
// and now that path too can fill all four ceilings, so a slow setup
|
||||
// must still leave the O4 gates time to run.
|
||||
['hosted-harness-mysql', 294, 327, 33],
|
||||
['hosted-harness-mysql', 295, 328, 33],
|
||||
// Twenty-seven failover arms at 10 each; the allowance is ~13 minutes of
|
||||
// setup plus 2.
|
||||
['hosted-failover-e2e', 270, 285, 15],
|
||||
@@ -931,12 +931,14 @@ describe('Hosted real-process gates', () => {
|
||||
// minutes; the FG12d two-server release cell adds 32 s (2x its
|
||||
// measured 16.2 s), moving the fork to 11536 and the step to 203
|
||||
// minutes; the FG12d detach-unknown cell adds 55 s (2x its measured
|
||||
// 27.4 s), moving the fork to 11591 and the step to 204 minutes until
|
||||
// 27.4 s), moving the fork to 11591 and the step to 204 minutes; the
|
||||
// FG12d stopped-worker close cell adds 67 s (2x its measured 33.1 s),
|
||||
// moving the fork to 11658 and the step to 205 minutes until
|
||||
// CI measures it: the relational check above tolerates anything up to
|
||||
// step*60, so the value is pinned literally here — an unnoticed raise
|
||||
// has no other red to land on.
|
||||
if (profile === 'hosted-harness-mysql') {
|
||||
expect(forkSeconds).toBe(11591);
|
||||
expect(forkSeconds).toBe(11658);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
Reference in new issue
Block a user