mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-08 08:11:17 -10:00
fix(daemon): PL-004 Phase C R2 — yaml-package frontmatter parser + discriminating scan_interval scheduler test
Repairs guard BLOCK on R1 commit dd9820e:
Blocker 1 (discriminating scheduler regression for scan_interval_seconds):
R1 added the right isDue() shape but proved it only with an engine-only
active-wake test that used equal interval values (interval=30,
scan=30). A scheduler regression to interval_seconds-only would have
passed it. R2 adds four discriminating tests in
packages/daemon/test/watchdog-scheduler.test.ts:
- isDue uses scan_interval_seconds (=30) over interval_seconds (=600):
due after 45s elapsed.
- isDue uses scan_interval_seconds (=600) over interval_seconds (=30):
NOT due after 45s elapsed.
- runTickNow does NOT invoke policy nor write history during the
scan_interval_seconds not-due window (drives the scheduler boundary,
asserts deliver callback never called AND no history row written AND
last_evaluation_at unchanged).
- runTickNow DOES invoke policy when scan_interval_seconds elapsed even
with large interval_seconds (the inverse: confirms scan_interval can
be smaller than interval and still trigger).
Lesson filed at memory/feedback_poc_regression_must_discriminate.md:
when porting POC regressions, preserve discriminating param values and
exercise the layer under review.
Blocker 2 (artifact-pool YAML/frontmatter parity):
R1 used a local key/value parser that treated any unquoted colon in a
value as malformed. Real frontmatter contains ISO timestamps
(claimed_at: 2026-05-03T05:15:25Z) and URLs
(source_url: https://...) that would be wrongly excluded. R2 swaps
in the `yaml` package (already a daemon dep) so:
- Valid YAML scalars containing colons parse cleanly and the artifact
is included.
- Only true YAML parse failures (broken: value: still broken) cause
exclusion when include_malformed_frontmatter=false.
- ScannedArtifact.frontmatter widened from Record<string, string> to
Record<string, unknown> because YAML loaders return heterogeneous
scalars (string, Date, number, etc.).
- statusFromFrontmatter() coerces non-string status scalars (Date,
number) to string so include_statuses comparisons stay stable.
- sourceKeyFor() coerces non-string key_field scalars similarly.
Three new POC-parity regression tests in
packages/daemon/test/watchdog-policies.test.ts:
- includes ready artifacts with ISO timestamp frontmatter.
- includes ready artifacts with URL frontmatter.
- includes ready artifacts with quoted-string frontmatter containing colons.
Plus the existing malformed-frontmatter exclusion test (broken: value:
still broken) continues to pass: that fixture is genuine invalid YAML.
Verification:
- tsc clean for daemon and cli.
- Per-package vitest baseline under explicit OPENRIG_HOME +
OPENRIG_DB=:memory: temp env:
- packages/daemon: 2492 / 2492 pass (was 2485 in R1; +7 new
watchdog tests for R2: 4 scheduler + 3 YAML-parity).
- packages/cli: 7 / 7 watchdog tests pass.
Boundaries preserved (R2):
- Worktree-only on phase-c-coordination-primitive-system; main untouched.
- No-touch surfaces unchanged: Phase A/B/Lifecycle/migrations 001-030/
workflow runtime/progress-rail/POC services-watchdog state.
- workflow-keepalive still rejected at register with policy_deferred_to_phase_d.
- No fleet sweep, no protected-host mutation, no external spend, no
destructive lifecycle, no VM-boundary actions, no remote push, no
tag, no publish.
- Three policy enum values unchanged: periodic-reminder,
artifact-pool-ready, edge-artifact-required.
- R1 in-place migration extension (actionable + last_actionable_at on
031) unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
1 parent
dd9820eb2d
commit
1a3b313f0d
3 files changed
+207
-41
No files matched your search
@@ -15,6 +15,7 @@
|
||||
|
||||
import { readdirSync, readFileSync, statSync } from "node:fs";
|
||||
import { basename, join } from "node:path";
|
||||
import { parse as parseYaml } from "yaml";
|
||||
|
||||
const DEFAULT_EXTENSIONS = [".md"];
|
||||
const DEFAULT_IGNORE_NAMES = ["README.md", ".DS_Store"];
|
||||
@@ -50,8 +51,13 @@ export interface ScannedArtifact {
|
||||
pool_path: string;
|
||||
/** Full file content (used by edge-artifact-required body-match). */
|
||||
raw: string;
|
||||
/** Parsed YAML frontmatter (top-level keys only). */
|
||||
frontmatter: Record<string, string>;
|
||||
/**
|
||||
* Parsed YAML frontmatter (top-level keys). Values are kept as `unknown`
|
||||
* because YAML scalars are heterogeneous: timestamps parse to Date,
|
||||
* URLs to string, numbers to number, etc. Frontmatter consumers in
|
||||
* the policy layer (status filter, key_field lookup) coerce to string.
|
||||
*/
|
||||
frontmatter: Record<string, unknown>;
|
||||
/** Parse error message when include_malformed_frontmatter=true; else null. */
|
||||
frontmatter_parse_error: string | null;
|
||||
/** Convenience accessor for frontmatter.status; null when absent. */
|
||||
@@ -65,14 +71,17 @@ const FRONTMATTER_OPEN = "---\n";
|
||||
* Returns { raw, frontmatter, parseError } so callers can decide whether
|
||||
* to include malformed artifacts.
|
||||
*
|
||||
* Detection of malformed frontmatter mirrors POC behavior: lines that
|
||||
* fail the `key: value` shape after the opener are treated as parse
|
||||
* errors (rather than silently dropped) so include_malformed_frontmatter
|
||||
* gating works as POC tests expect.
|
||||
* R2 fix (guard blocker 2): use the `yaml` package (already a daemon
|
||||
* dep) rather than a local key/value parser. Mirrors POC's
|
||||
* `lib/policies/artifact-pool.js:21-30` which delegates to the shared
|
||||
* YAML loader. Valid YAML scalars containing colons (ISO timestamps,
|
||||
* URLs) parse cleanly. Only true YAML parse failures (e.g.,
|
||||
* `broken: value: still broken` which is genuinely invalid YAML) cause
|
||||
* exclusion when `include_malformed_frontmatter` is unset.
|
||||
*/
|
||||
function readFrontmatter(filePath: string): {
|
||||
raw: string;
|
||||
frontmatter: Record<string, string>;
|
||||
frontmatter: Record<string, unknown>;
|
||||
frontmatter_parse_error: string | null;
|
||||
} {
|
||||
const raw = readFileSync(filePath, "utf-8");
|
||||
@@ -84,42 +93,26 @@ function readFrontmatter(filePath: string): {
|
||||
return { raw, frontmatter: {}, frontmatter_parse_error: null };
|
||||
}
|
||||
const block = raw.slice(FRONTMATTER_OPEN.length, endIdx);
|
||||
const fm: Record<string, string> = {};
|
||||
let parseError: string | null = null;
|
||||
for (const line of block.split("\n")) {
|
||||
if (line.trim().length === 0) continue;
|
||||
const colonIdx = line.indexOf(":");
|
||||
if (colonIdx === -1) {
|
||||
parseError = `frontmatter line missing ':' (${line.slice(0, 40)})`;
|
||||
break;
|
||||
}
|
||||
const key = line.slice(0, colonIdx).trim();
|
||||
const valueRaw = line.slice(colonIdx + 1);
|
||||
let value = valueRaw.trim();
|
||||
// Detect malformed values like 'broken: value: still broken' where the
|
||||
// value side itself contains another `:` outside quotes. POC treats
|
||||
// these as parse errors so half-written drafts do not slip through.
|
||||
const stripped = value.replace(/^"(.*)"$/, "$1").replace(/^'(.*)'$/, "$1");
|
||||
if (
|
||||
stripped !== value &&
|
||||
stripped.length === value.length - 2
|
||||
) {
|
||||
// Was quoted; treat as a clean string regardless of inner colons.
|
||||
value = stripped;
|
||||
} else if (value.includes(":")) {
|
||||
parseError = `frontmatter value contains unescaped ':' for key '${key}'`;
|
||||
break;
|
||||
}
|
||||
if (!key) {
|
||||
parseError = "frontmatter line missing key";
|
||||
break;
|
||||
}
|
||||
fm[key] = value;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = parseYaml(block);
|
||||
} catch (err) {
|
||||
return {
|
||||
raw,
|
||||
frontmatter: {},
|
||||
frontmatter_parse_error: err instanceof Error ? err.message : "frontmatter parse error",
|
||||
};
|
||||
}
|
||||
// YAML parsed but produced a non-object root (string, array, null) —
|
||||
// treat as empty frontmatter, NOT a parse error. Matches POC behavior
|
||||
// (lib/policies/artifact-pool.js:35-38): non-object root yields {}.
|
||||
if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||||
return { raw, frontmatter: {}, frontmatter_parse_error: null };
|
||||
}
|
||||
return {
|
||||
raw,
|
||||
frontmatter: parseError ? {} : fm,
|
||||
frontmatter_parse_error: parseError,
|
||||
frontmatter: parsed as Record<string, unknown>,
|
||||
frontmatter_parse_error: null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -160,6 +153,20 @@ function statusAllowed(status: string | null, pool: ArtifactPoolSpec): boolean {
|
||||
return status !== null && include.includes(status);
|
||||
}
|
||||
|
||||
/**
|
||||
* Coerce a YAML-parsed frontmatter status field to a comparable string.
|
||||
* YAML loaders may parse unquoted statuses as identifiers (string),
|
||||
* but a status like `2026-05-03` would parse as Date. We coerce to ISO
|
||||
* for non-string scalars so include_statuses comparisons remain stable.
|
||||
*/
|
||||
function statusFromFrontmatter(fm: Record<string, unknown>): string | null {
|
||||
const v = fm.status;
|
||||
if (v === undefined || v === null) return null;
|
||||
if (typeof v === "string") return v;
|
||||
if (v instanceof Date) return v.toISOString();
|
||||
return String(v);
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan one or more artifact pools per spec(s). Mirrors POC
|
||||
* `scanArtifactPools(pools)`. Returns the flat union of matches across
|
||||
@@ -199,7 +206,7 @@ export async function scanArtifactPools(
|
||||
if (parsed.frontmatter_parse_error && !pool.include_malformed_frontmatter) {
|
||||
continue;
|
||||
}
|
||||
const status = parsed.frontmatter.status ?? null;
|
||||
const status = statusFromFrontmatter(parsed.frontmatter);
|
||||
if (!statusAllowed(status, pool)) continue;
|
||||
out.push({
|
||||
path: filePath,
|
||||
@@ -249,11 +256,14 @@ export function formatArtifactList(artifacts: ScannedArtifact[], maxItems: numbe
|
||||
/**
|
||||
* Compute the source-key for an artifact. Mirrors POC sourceKeyFor:
|
||||
* prefer frontmatter[keyField], else basename sans .md extension.
|
||||
* Coerces non-string scalars (Date, number) to string for comparison.
|
||||
*/
|
||||
export function sourceKeyFor(artifact: ScannedArtifact, keyField: string): string {
|
||||
const value = artifact.frontmatter[keyField];
|
||||
if (value === undefined || value === null || value === "") {
|
||||
return basename(artifact.path).replace(/\.md$/, "");
|
||||
}
|
||||
if (typeof value === "string") return value;
|
||||
if (value instanceof Date) return value.toISOString();
|
||||
return String(value);
|
||||
}
|
||||
@@ -198,6 +198,61 @@ describe("artifactPoolReadyPolicy (POC contract)", () => {
|
||||
expect(out.message).not.toContain("malformed.md");
|
||||
});
|
||||
|
||||
// R2 fix (guard blocker 2): valid YAML scalars containing colons
|
||||
// (ISO timestamps, URLs) MUST parse cleanly via the `yaml` package
|
||||
// and the artifact MUST be included. The R1 local parser excluded
|
||||
// these as "malformed" because of the inner colons.
|
||||
it("includes ready artifacts with ISO timestamp frontmatter (POC YAML parity)", async () => {
|
||||
writeFileSync(
|
||||
join(tmp, "with-timestamp.md"),
|
||||
"---\nstatus: ready\nclaimed_at: 2026-05-03T05:15:25Z\n---\n# Body\n",
|
||||
);
|
||||
writeFileSync(
|
||||
join(tmp, "malformed.md"),
|
||||
"---\nstatus: ready\nbroken: value: still broken\n---\n",
|
||||
);
|
||||
const out = await artifactPoolReadyPolicy.evaluate(
|
||||
makeJob({
|
||||
context: { pools: [{ path: tmp, include_statuses: ["ready"] }] },
|
||||
}),
|
||||
);
|
||||
expect(out.action).toBe("send");
|
||||
if (out.action !== "send") return;
|
||||
expect(out.message).toContain("with-timestamp.md");
|
||||
expect(out.message).not.toContain("malformed.md");
|
||||
expect(out.message).toMatch(/has 1 actionable/);
|
||||
});
|
||||
|
||||
it("includes ready artifacts with URL frontmatter (POC YAML parity)", async () => {
|
||||
writeFileSync(
|
||||
join(tmp, "with-url.md"),
|
||||
"---\nstatus: ready\nsource_url: https://example.org/path\n---\n# Body\n",
|
||||
);
|
||||
const out = await artifactPoolReadyPolicy.evaluate(
|
||||
makeJob({
|
||||
context: { pools: [{ path: tmp, include_statuses: ["ready"] }] },
|
||||
}),
|
||||
);
|
||||
expect(out.action).toBe("send");
|
||||
if (out.action !== "send") return;
|
||||
expect(out.message).toContain("with-url.md");
|
||||
});
|
||||
|
||||
it("includes ready artifacts with quoted-string frontmatter containing colons", async () => {
|
||||
writeFileSync(
|
||||
join(tmp, "with-quoted-colon.md"),
|
||||
"---\nstatus: ready\ntitle: \"Phase C: things\"\n---\n",
|
||||
);
|
||||
const out = await artifactPoolReadyPolicy.evaluate(
|
||||
makeJob({
|
||||
context: { pools: [{ path: tmp, include_statuses: ["ready"] }] },
|
||||
}),
|
||||
);
|
||||
expect(out.action).toBe("send");
|
||||
if (out.action !== "send") return;
|
||||
expect(out.message).toContain("with-quoted-colon.md");
|
||||
});
|
||||
|
||||
it("supports configured ignore_names (POC parity)", async () => {
|
||||
writeFileSync(join(tmp, "skip-me.md"), "---\nstatus: ready\n---\n");
|
||||
writeFileSync(join(tmp, "include-me.md"), "---\nstatus: ready\n---\n");
|
||||
|
||||
@@ -194,6 +194,107 @@ describe("WatchdogScheduler (PL-004 Phase C)", () => {
|
||||
expect(sched.isRunning()).toBe(false);
|
||||
});
|
||||
|
||||
// R2 fix (guard blocker 1): the scheduler MUST consult
|
||||
// scan_interval_seconds when set; interval_seconds is a fallback only.
|
||||
// These tests use distinct values so a regression that uses
|
||||
// interval_seconds will fail (unlike R1's equal-valued active-wake test).
|
||||
it("isDue uses scan_interval_seconds (=30) over interval_seconds (=600) — due after 45s elapsed", () => {
|
||||
const job = jobsRepo.register({
|
||||
policy: "periodic-reminder",
|
||||
specYaml: "policy: periodic-reminder\ntarget:\n session: a@rig\nmessage: x\n",
|
||||
targetSession: "a@rig",
|
||||
intervalSeconds: 600,
|
||||
scanIntervalSeconds: 30,
|
||||
registeredBySession: "ops@kernel",
|
||||
});
|
||||
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
|
||||
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
|
||||
const updated = jobsRepo.getByIdOrThrow(job.jobId);
|
||||
expect(isDue(updated, Date.now())).toBe(true);
|
||||
});
|
||||
|
||||
it("isDue uses scan_interval_seconds (=600) over interval_seconds (=30) — NOT due after 45s elapsed", () => {
|
||||
const job = jobsRepo.register({
|
||||
policy: "periodic-reminder",
|
||||
specYaml: "policy: periodic-reminder\ntarget:\n session: a@rig\nmessage: x\n",
|
||||
targetSession: "a@rig",
|
||||
intervalSeconds: 30,
|
||||
scanIntervalSeconds: 600,
|
||||
registeredBySession: "ops@kernel",
|
||||
});
|
||||
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
|
||||
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
|
||||
const updated = jobsRepo.getByIdOrThrow(job.jobId);
|
||||
expect(isDue(updated, Date.now())).toBe(false);
|
||||
});
|
||||
|
||||
it("runTickNow does NOT invoke policy nor write history during scan_interval_seconds not-due window", async () => {
|
||||
// scan_interval_seconds=600 with interval_seconds=30 (the discriminating
|
||||
// pair: a scheduler that ignores scan_interval_seconds would be due
|
||||
// after 30s and would invoke the policy + write history). Test asserts
|
||||
// policy is NOT invoked through the scheduler boundary AND that no
|
||||
// sent history row was written.
|
||||
let policyCalled = 0;
|
||||
const job = jobsRepo.register({
|
||||
policy: "periodic-reminder",
|
||||
specYaml:
|
||||
"policy: periodic-reminder\ntarget:\n session: counted@rig\nmessage: counted\n",
|
||||
targetSession: "counted@rig",
|
||||
intervalSeconds: 30,
|
||||
scanIntervalSeconds: 600,
|
||||
registeredBySession: "ops@kernel",
|
||||
});
|
||||
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
|
||||
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
|
||||
const engine = new WatchdogPolicyEngine({
|
||||
jobsRepo,
|
||||
historyLog: log,
|
||||
eventBus: bus,
|
||||
deliver: async (req) => {
|
||||
policyCalled += 1;
|
||||
deliveries.push(req);
|
||||
return { status: "ok" };
|
||||
},
|
||||
});
|
||||
const sched = new WatchdogScheduler({ jobsRepo, policyEngine: engine });
|
||||
const historyCountBefore = log.countForJob(job.jobId);
|
||||
await sched.runTickNow();
|
||||
expect(policyCalled).toBe(0);
|
||||
expect(deliveries.length).toBe(0);
|
||||
expect(log.countForJob(job.jobId)).toBe(historyCountBefore);
|
||||
expect(jobsRepo.getByIdOrThrow(job.jobId).lastEvaluationAt).toBe(fortyFiveSecondsAgo);
|
||||
});
|
||||
|
||||
it("runTickNow DOES invoke policy when scan_interval_seconds elapsed even if interval_seconds large", async () => {
|
||||
let policyCalled = 0;
|
||||
const job = jobsRepo.register({
|
||||
policy: "periodic-reminder",
|
||||
specYaml:
|
||||
"policy: periodic-reminder\ntarget:\n session: counted@rig\nmessage: counted\n",
|
||||
targetSession: "counted@rig",
|
||||
intervalSeconds: 600,
|
||||
scanIntervalSeconds: 30,
|
||||
registeredBySession: "ops@kernel",
|
||||
});
|
||||
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
|
||||
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
|
||||
const engine = new WatchdogPolicyEngine({
|
||||
jobsRepo,
|
||||
historyLog: log,
|
||||
eventBus: bus,
|
||||
deliver: async (req) => {
|
||||
policyCalled += 1;
|
||||
deliveries.push(req);
|
||||
return { status: "ok" };
|
||||
},
|
||||
});
|
||||
const sched = new WatchdogScheduler({ jobsRepo, policyEngine: engine });
|
||||
await sched.runTickNow();
|
||||
expect(policyCalled).toBe(1);
|
||||
expect(deliveries).toEqual([{ targetSession: "counted@rig", message: "counted" }]);
|
||||
expect(log.countForJob(job.jobId)).toBe(1);
|
||||
});
|
||||
|
||||
it("recovers schedule across restart via SQLite (new repo + new scheduler picks up active jobs)", async () => {
|
||||
jobsRepo.register({
|
||||
policy: "periodic-reminder",
|
||||
|
||||
Reference in new issue
Block a user