mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
ci(e2e): time-box the artifact download retry and the two uncapped legs (#15917)
Each leg downloads the shared build archive twice: once, and again from the retry that only fires when the first attempt failed. Only the first attempt carried timeout-minutes, so a retry that stalls held its runner with no ceiling at all — up to GitHub's 360-minute default. Run 37119220504 (#13284) already showed both attempts lost inside one window on a macOS shard while its sibling downloaded the same artifact. Two limbs: - Give the four 'Download build artifact (retry)' steps timeout-minutes: 10. The value is not new: the four first-attempt downloads in the same jobs already pin 10. - Give e2e-test-macos and isolated-nightly a job-level timeout-minutes: 60. The two other artifact-consuming legs (e2e-test-linux, e2e-interactive-opentui) already use 60, and 60 is also the ceiling the shard-retry budget gate in run-e2e-tests.sh takes its 25-minute reserve out of. `build` keeps its own 30 — it produces the archive rather than consuming it. web-shell-browser-regression has the same missing job ceiling but is left alone: it is outside this finding's scope and does not consume the build artifact. The retry steps keep their absence of continue-on-error — the first attempt's continue-on-error is what makes the retry reachable, and copying it onto the retry would let a stalled download pass silently into the unpack step. isolated-nightly's job-level continue-on-error is untouched, so its 60-minute ceiling frees the runner without changing the leg's conclusion. Pinned in scripts/tests/e2e-workflow.test.js: both limbs, the retry's visible failure, and a regression guard over the already-landed values (build 30, the two consuming legs 60, the four first-attempt steps 10). Refs #12132 Patrol-Run: qwen-issue-patrol/jmvuv88frpw Co-authored-by: yiliang114 <[email protected]> Co-authored-by: Qwen-Coder <[email protected]>
This commit is contained in:
2 files changed
+80
No files matched your search
@@ -385,6 +385,7 @@ jobs:
|
||||
- name: 'Download build artifact (retry)'
|
||||
if: "${{ steps.download-build.outcome == 'failure' }}"
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
timeout-minutes: 10
|
||||
with:
|
||||
name: 'e2e-build'
|
||||
path: '${{ runner.temp }}/e2e-build'
|
||||
@@ -465,6 +466,7 @@ jobs:
|
||||
e2e-test-macos:
|
||||
name: 'E2E Test - macOS - shard ${{ matrix.shard }}'
|
||||
runs-on: 'macos-latest'
|
||||
timeout-minutes: 60
|
||||
needs:
|
||||
- 'build'
|
||||
# Skip on fork PRs (no secrets) — see e2e-test-linux above.
|
||||
@@ -554,6 +556,7 @@ jobs:
|
||||
- name: 'Download build artifact (retry)'
|
||||
if: "${{ steps.download-build.outcome == 'failure' }}"
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
timeout-minutes: 10
|
||||
with:
|
||||
name: 'e2e-build'
|
||||
path: '${{ runner.temp }}/e2e-build'
|
||||
@@ -671,6 +674,7 @@ jobs:
|
||||
- name: 'Download build artifact (retry)'
|
||||
if: "${{ steps.download-build.outcome == 'failure' }}"
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
timeout-minutes: 10
|
||||
with:
|
||||
name: 'e2e-build'
|
||||
path: '${{ runner.temp }}/e2e-build'
|
||||
@@ -699,6 +703,7 @@ jobs:
|
||||
isolated-nightly:
|
||||
name: '${{ matrix.label }} (nightly)'
|
||||
runs-on: 'ubuntu-latest'
|
||||
timeout-minutes: 60
|
||||
needs:
|
||||
- 'build'
|
||||
if: |-
|
||||
@@ -785,6 +790,7 @@ jobs:
|
||||
- name: 'Download build artifact (retry)'
|
||||
if: "${{ steps.download-build.outcome == 'failure' }}"
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
timeout-minutes: 10
|
||||
with:
|
||||
name: 'e2e-build'
|
||||
path: '${{ runner.temp }}/e2e-build'
|
||||
|
||||
@@ -1342,6 +1342,80 @@ describe('e2e workflow', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('artifact download time-boxing', () => {
|
||||
// #12132: each leg downloads the shared archive twice — once, and again
|
||||
// from the retry that only fires when the first attempt failed. Only the
|
||||
// first attempt was time-boxed, so a stalling retry held its runner with
|
||||
// no ceiling at all, and two legs had no job-level ceiling either.
|
||||
const legs = [
|
||||
'e2e-test-linux',
|
||||
'e2e-test-macos',
|
||||
'e2e-interactive-opentui',
|
||||
'isolated-nightly',
|
||||
];
|
||||
const downloads = (leg) =>
|
||||
yml.jobs[leg].steps.filter((step) =>
|
||||
String(step.name).startsWith('Download build artifact'),
|
||||
);
|
||||
|
||||
it('caps every artifact-consuming leg at the job level', () => {
|
||||
// 60 is the ceiling e2e-test-linux and e2e-interactive-opentui already
|
||||
// carried; the other two legs consume the same archive on the same
|
||||
// runners, so they get that budget instead of a new number.
|
||||
for (const leg of legs) {
|
||||
expect(yml.jobs[leg]['timeout-minutes'], leg).toBe(60);
|
||||
}
|
||||
});
|
||||
|
||||
it('time-boxes the retry as tightly as the first attempt', () => {
|
||||
// 10 is the value every first-attempt download already pins. Without it
|
||||
// the retry — the step that exists precisely because the first attempt
|
||||
// misbehaved — was the only unbounded step in the download leg.
|
||||
for (const leg of legs) {
|
||||
const steps = downloads(leg);
|
||||
expect(
|
||||
steps.map((step) => step.name),
|
||||
leg,
|
||||
).toEqual([
|
||||
'Download build artifact',
|
||||
'Download build artifact (retry)',
|
||||
]);
|
||||
for (const step of steps) {
|
||||
expect(step['timeout-minutes'], `${leg}/${step.name}`).toBe(10);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps the retry step red when the retry itself fails', () => {
|
||||
// The first attempt's continue-on-error is what makes the retry
|
||||
// reachable. Copying it onto the retry would leave a stalled download
|
||||
// invisible: the leg would go on to unpack a missing archive and report
|
||||
// a failure with the wrong cause.
|
||||
for (const leg of legs) {
|
||||
const retry = downloads(leg)[1];
|
||||
expect(retry.name, leg).toBe('Download build artifact (retry)');
|
||||
expect(retry['continue-on-error']).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('leaves the already-landed ceilings where they are', () => {
|
||||
// Regression guard. `build` produces the archive on a 30-minute budget
|
||||
// while the legs consume it on 60, and 60 is also the number the
|
||||
// shard-retry budget gate in run-e2e-tests.sh takes its 25-minute
|
||||
// reserve out of. A later pass that "unifies" these values would move
|
||||
// either the producer's ceiling or the retry arithmetic with every
|
||||
// other witness green.
|
||||
expect(yml.jobs.build['timeout-minutes']).toBe(30);
|
||||
expect(yml.jobs['e2e-test-linux']['timeout-minutes']).toBe(60);
|
||||
expect(yml.jobs['e2e-interactive-opentui']['timeout-minutes']).toBe(60);
|
||||
for (const leg of legs) {
|
||||
const first = downloads(leg)[0];
|
||||
expect(first['timeout-minutes'], leg).toBe(10);
|
||||
expect(first['continue-on-error'], leg).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('one build for every leg', () => {
|
||||
// Each leg used to build and bundle on its own runner — 4–8 minutes on a
|
||||
// hosted VM, 10–17 on a busy pool host, eleven times per run. The `build`
|
||||
|
||||
Reference in new issue
Block a user