Files
14f8d6539a ci(frontend): shard integration tests with verified coverage (#14280)
### Why / What / How

Frontend integration tests previously ran as one serial job behind a
shared setup job. This PR runs the same Vitest suite as four independent
shards, keeps each shard's coverage and JUnit evidence, and rejects
missing, malformed, or inconsistent reports before coverage upload.

All frontend jobs use standard GitHub-hosted `ubuntu-latest` runners.
The separate Full-stack CI workflow and its existing `big-boi` E2E job
are unchanged; this PR does not multiply paid E2E jobs or move shards
onto that runner.

### Changes 🏗️

- Run four native Vitest shards with coverage collection, per-shard
artifacts, and mandatory JUnit and Cobertura validation. Each active job
installs its own dependencies; removing the shared setup job removes the
serial dependency.
- Enforce the standing frontend no-skip policy. Skipped or omitted tests
fail validation; legitimate policy exceptions require review rather than
silent quarantine.
- Validate Cobertura structure, counters, nested packages, line
evidence, and rounding before Codecov upload. Failed validation still
preserves available artifacts.
- Add shared command/unittest JUnit reporters and their regressions.
Exact skip IDs, discovery crashes, fixture errors alongside passing
tests, repeated subtest failures, and later cleanup errors are covered.
Every failure is retained in the report, with duration counted once.
- Make report-write failures return a clear nonzero result and
diagnostic. Command failure/spawn exit statuses remain intact even when
the report cannot be written.
- Keep Chromatic disabled. Re-enabling it must require `(github.ref ==
'refs/heads/dev' || github.base_ref == 'dev')` and a paths-filter match
on `autogpt_platform/frontend/src/components/**`; the removed serial
setup job should not be restored just for that gate.

### Validation

- Current cleanup: 55 local tests pass (37 CI helpers and 18 reporter
tests). The report-write test covers ten command/unittest and
invalid-destination combinations.
- Three deliberately broken reporter variants are caught: any nonempty
allowlist permitting all skips, discovery crashes reporting success, and
fixture-error records being omitted. Two new subtest/cleanup regressions
failed before their source fix and pass afterward. All ten report-write
cases failed before their fix and pass afterward.
- Whole CI-helper/appliance-test Black formatting, diff checks, and
normal commit/push hooks pass. The cleanup's workflow change is comments
only; parsed frontend workflow configuration is identical to the
approved head, and Full-stack CI is byte-for-byte unchanged.
- Actionlint reports only the intentional existing constant-false
Chromatic gate; its earlier removal request was explicitly withdrawn
after baseline behavior was explained. No lint rules or CI gates were
weakened.
- Historical measurement from the earlier shard implementation: wall
time 14m07s → 4m32s and standard runner usage 16.58 → 20.01 minutes
(+21%). These are historical samples, not a new billing guarantee.
Actual current-head hosted CI and fresh review remain required.
- Full local frontend/application and live Docker E2E suites were not
rerun for this CI-helper cleanup; hosted workflows validate the
application matrix.

### Scope and provenance

Targets `dev`; former prerequisite #14279 is merged. Downstream CI PRs
and #14114 are outside this PR. The current diff includes the previously
consolidated frontend validators/shared reporters; earlier historical
review and CI claims are not substituted for fresh-head gates.

### Agents and large language models used

- Codex with GPT-6: review-comment validation, regression tests,
diagnostics fixes, workflow/run inspection, and PR maintenance.
- Earlier history credits Codex (model unavailable), Codex with
`gpt-daybreak-blue-latest` for independent review, and Claude Opus 5
(originating platform not recorded). Existing attribution is preserved.

### Checklist 📋

#### For code changes:

- [x] Changes and ownership are listed.
- [x] Test plan and validation limitations are described.
- [x] Relevant local CI-helper tests, formatting, and normal hooks pass.
- [ ] Current-head hosted workflow checks pass.
- [ ] Fresh independent review covers the current head.

#### For configuration changes:

- [x] Existing environment and Compose configuration remain compatible.
- [x] Runner, sharding, report-validation, and disabled Chromatic
behavior are listed above.

---------

Co-authored-by: Claude Opus 5 <[email protected]>
Co-authored-by: Reinier van der Leer <[email protected]>
Co-authored-by: gpt-daybreak-blue-latest (Codex) <[email protected]>
2026-09-14 04:31:52 +00:00

98 lines
2.7 KiB
Python

#!/usr/bin/env python3
import argparse
import subprocess
import sys
import time
import xml.etree.ElementTree as ET
from pathlib import Path
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--name", required=True)
parser.add_argument("--classname", required=True)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("command", nargs=argparse.REMAINDER)
args = parser.parse_args()
if args.command[:1] == ["--"]:
args.command = args.command[1:]
if not args.command:
parser.error("a command is required after --")
return args
def write_junit(
path: Path,
name: str,
classname: str,
command: list[str],
returncode: int | None,
duration: float,
spawn_error: str | None = None,
) -> None:
suite = ET.Element(
"testsuite",
{
"name": classname,
"tests": "1",
"failures": "1" if returncode and spawn_error is None else "0",
"errors": "1" if spawn_error is not None else "0",
"skipped": "0",
"time": f"{duration:.6f}",
},
)
case = ET.SubElement(
suite,
"testcase",
{"classname": classname, "name": name, "time": f"{duration:.6f}"},
)
if spawn_error is not None:
error = ET.SubElement(
case,
"error",
{"message": "command could not be started"},
)
error.text = spawn_error
elif returncode:
failure = ET.SubElement(
case,
"failure",
{"message": f"command exited with status {returncode}"},
)
failure.text = " ".join(command)
path.parent.mkdir(parents=True, exist_ok=True)
ET.ElementTree(suite).write(path, encoding="utf-8", xml_declaration=True)
def main() -> int:
args = parse_args()
started_at = time.monotonic()
spawn_error = None
try:
completed = subprocess.run(args.command, check=False)
returncode = completed.returncode
except OSError as error:
spawn_error = f"{type(error).__name__}: {error}"
returncode = 127
duration = time.monotonic() - started_at
try:
write_junit(
args.output,
args.name,
args.classname,
args.command,
None if spawn_error is not None else returncode,
duration,
spawn_error=spawn_error,
)
except OSError as error:
print(f"Failed to write JUnit report {args.output}: {error}", file=sys.stderr)
return returncode or 1
return returncode
if __name__ == "__main__":
raise SystemExit(main())