fix(daemon): PL-004 Phase C R2 — yaml-package frontmatter parser + discriminating scan_interval scheduler test

Repairs guard BLOCK on R1 commit dd9820e:

Blocker 1 (discriminating scheduler regression for scan_interval_seconds):

R1 added the right isDue() shape but proved it only with an engine-only
active-wake test that used equal interval values (interval=30,
scan=30). A scheduler regression to interval_seconds-only would have
passed it. R2 adds four discriminating tests in
packages/daemon/test/watchdog-scheduler.test.ts:

- isDue uses scan_interval_seconds (=30) over interval_seconds (=600):
  due after 45s elapsed.
- isDue uses scan_interval_seconds (=600) over interval_seconds (=30):
  NOT due after 45s elapsed.
- runTickNow does NOT invoke policy nor write history during the
  scan_interval_seconds not-due window (drives the scheduler boundary,
  asserts deliver callback never called AND no history row written AND
  last_evaluation_at unchanged).
- runTickNow DOES invoke policy when scan_interval_seconds elapsed even
  with large interval_seconds (the inverse: confirms scan_interval can
  be smaller than interval and still trigger).

Lesson filed at memory/feedback_poc_regression_must_discriminate.md:
when porting POC regressions, preserve discriminating param values and
exercise the layer under review.

Blocker 2 (artifact-pool YAML/frontmatter parity):

R1 used a local key/value parser that treated any unquoted colon in a
value as malformed. Real frontmatter contains ISO timestamps
(claimed_at: 2026-05-03T05:15:25Z) and URLs
(source_url: https://...) that would be wrongly excluded. R2 swaps
in the `yaml` package (already a daemon dep) so:

- Valid YAML scalars containing colons parse cleanly and the artifact
  is included.
- Only true YAML parse failures (broken: value: still broken) cause
  exclusion when include_malformed_frontmatter=false.
- ScannedArtifact.frontmatter widened from Record<string, string> to
  Record<string, unknown> because YAML loaders return heterogeneous
  scalars (string, Date, number, etc.).
- statusFromFrontmatter() coerces non-string status scalars (Date,
  number) to string so include_statuses comparisons stay stable.
- sourceKeyFor() coerces non-string key_field scalars similarly.

Three new POC-parity regression tests in
packages/daemon/test/watchdog-policies.test.ts:

- includes ready artifacts with ISO timestamp frontmatter.
- includes ready artifacts with URL frontmatter.
- includes ready artifacts with quoted-string frontmatter containing colons.

Plus the existing malformed-frontmatter exclusion test (broken: value:
still broken) continues to pass: that fixture is genuine invalid YAML.

Verification:
   - tsc clean for daemon and cli.
   - Per-package vitest baseline under explicit OPENRIG_HOME +
     OPENRIG_DB=:memory: temp env:
     - packages/daemon: 2492 / 2492 pass (was 2485 in R1; +7 new
       watchdog tests for R2: 4 scheduler + 3 YAML-parity).
     - packages/cli: 7 / 7 watchdog tests pass.

Boundaries preserved (R2):
   - Worktree-only on phase-c-coordination-primitive-system; main untouched.
   - No-touch surfaces unchanged: Phase A/B/Lifecycle/migrations 001-030/
     workflow runtime/progress-rail/POC services-watchdog state.
   - workflow-keepalive still rejected at register with policy_deferred_to_phase_d.
   - No fleet sweep, no protected-host mutation, no external spend, no
     destructive lifecycle, no VM-boundary actions, no remote push, no
     tag, no publish.
   - Three policy enum values unchanged: periodic-reminder,
     artifact-pool-ready, edge-artifact-required.
   - R1 in-place migration extension (actionable + last_actionable_at on
     031) unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
mvschwarzandClaude Opus 4.7 committed 2026-05-03 00:53:16 -07:00
1 parent dd9820eb2d
commit 1a3b313f0d
3 files changed
+207 -41

No files matched your search

@@ -15,6 +15,7 @@
import { readdirSync, readFileSync, statSync } from "node:fs";
import { basename, join } from "node:path";
import { parse as parseYaml } from "yaml";
const DEFAULT_EXTENSIONS = [".md"];
const DEFAULT_IGNORE_NAMES = ["README.md", ".DS_Store"];
@@ -50,8 +51,13 @@ export interface ScannedArtifact {
pool_path: string;
/** Full file content (used by edge-artifact-required body-match). */
raw: string;
/** Parsed YAML frontmatter (top-level keys only). */
frontmatter: Record<string, string>;
/**
* Parsed YAML frontmatter (top-level keys). Values are kept as `unknown`
* because YAML scalars are heterogeneous: timestamps parse to Date,
* URLs to string, numbers to number, etc. Frontmatter consumers in
* the policy layer (status filter, key_field lookup) coerce to string.
*/
frontmatter: Record<string, unknown>;
/** Parse error message when include_malformed_frontmatter=true; else null. */
frontmatter_parse_error: string | null;
/** Convenience accessor for frontmatter.status; null when absent. */
@@ -65,14 +71,17 @@ const FRONTMATTER_OPEN = "---\n";
* Returns { raw, frontmatter, parseError } so callers can decide whether
* to include malformed artifacts.
*
* Detection of malformed frontmatter mirrors POC behavior: lines that
* fail the `key: value` shape after the opener are treated as parse
* errors (rather than silently dropped) so include_malformed_frontmatter
* gating works as POC tests expect.
* R2 fix (guard blocker 2): use the `yaml` package (already a daemon
* dep) rather than a local key/value parser. Mirrors POC's
* `lib/policies/artifact-pool.js:21-30` which delegates to the shared
* YAML loader. Valid YAML scalars containing colons (ISO timestamps,
* URLs) parse cleanly. Only true YAML parse failures (e.g.,
* `broken: value: still broken` which is genuinely invalid YAML) cause
* exclusion when `include_malformed_frontmatter` is unset.
*/
function readFrontmatter(filePath: string): {
raw: string;
frontmatter: Record<string, string>;
frontmatter: Record<string, unknown>;
frontmatter_parse_error: string | null;
} {
const raw = readFileSync(filePath, "utf-8");
@@ -84,42 +93,26 @@ function readFrontmatter(filePath: string): {
return { raw, frontmatter: {}, frontmatter_parse_error: null };
}
const block = raw.slice(FRONTMATTER_OPEN.length, endIdx);
const fm: Record<string, string> = {};
let parseError: string | null = null;
for (const line of block.split("\n")) {
if (line.trim().length === 0) continue;
const colonIdx = line.indexOf(":");
if (colonIdx === -1) {
parseError = `frontmatter line missing ':' (${line.slice(0, 40)})`;
break;
}
const key = line.slice(0, colonIdx).trim();
const valueRaw = line.slice(colonIdx + 1);
let value = valueRaw.trim();
// Detect malformed values like 'broken: value: still broken' where the
// value side itself contains another `:` outside quotes. POC treats
// these as parse errors so half-written drafts do not slip through.
const stripped = value.replace(/^"(.*)"$/, "$1").replace(/^'(.*)'$/, "$1");
if (
stripped !== value &&
stripped.length === value.length - 2
) {
// Was quoted; treat as a clean string regardless of inner colons.
value = stripped;
} else if (value.includes(":")) {
parseError = `frontmatter value contains unescaped ':' for key '${key}'`;
break;
}
if (!key) {
parseError = "frontmatter line missing key";
break;
}
fm[key] = value;
let parsed: unknown;
try {
parsed = parseYaml(block);
} catch (err) {
return {
raw,
frontmatter: {},
frontmatter_parse_error: err instanceof Error ? err.message : "frontmatter parse error",
};
}
// YAML parsed but produced a non-object root (string, array, null) —
// treat as empty frontmatter, NOT a parse error. Matches POC behavior
// (lib/policies/artifact-pool.js:35-38): non-object root yields {}.
if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
return { raw, frontmatter: {}, frontmatter_parse_error: null };
}
return {
raw,
frontmatter: parseError ? {} : fm,
frontmatter_parse_error: parseError,
frontmatter: parsed as Record<string, unknown>,
frontmatter_parse_error: null,
};
}
@@ -160,6 +153,20 @@ function statusAllowed(status: string | null, pool: ArtifactPoolSpec): boolean {
return status !== null && include.includes(status);
}
/**
* Coerce a YAML-parsed frontmatter status field to a comparable string.
* YAML loaders may parse unquoted statuses as identifiers (string),
* but a status like `2026-05-03` would parse as Date. We coerce to ISO
* for non-string scalars so include_statuses comparisons remain stable.
*/
function statusFromFrontmatter(fm: Record<string, unknown>): string | null {
const v = fm.status;
if (v === undefined || v === null) return null;
if (typeof v === "string") return v;
if (v instanceof Date) return v.toISOString();
return String(v);
}
/**
* Scan one or more artifact pools per spec(s). Mirrors POC
* `scanArtifactPools(pools)`. Returns the flat union of matches across
@@ -199,7 +206,7 @@ export async function scanArtifactPools(
if (parsed.frontmatter_parse_error && !pool.include_malformed_frontmatter) {
continue;
}
const status = parsed.frontmatter.status ?? null;
const status = statusFromFrontmatter(parsed.frontmatter);
if (!statusAllowed(status, pool)) continue;
out.push({
path: filePath,
@@ -249,11 +256,14 @@ export function formatArtifactList(artifacts: ScannedArtifact[], maxItems: numbe
/**
* Compute the source-key for an artifact. Mirrors POC sourceKeyFor:
* prefer frontmatter[keyField], else basename sans .md extension.
* Coerces non-string scalars (Date, number) to string for comparison.
*/
export function sourceKeyFor(artifact: ScannedArtifact, keyField: string): string {
const value = artifact.frontmatter[keyField];
if (value === undefined || value === null || value === "") {
return basename(artifact.path).replace(/\.md$/, "");
}
if (typeof value === "string") return value;
if (value instanceof Date) return value.toISOString();
return String(value);
}
@@ -198,6 +198,61 @@ describe("artifactPoolReadyPolicy (POC contract)", () => {
expect(out.message).not.toContain("malformed.md");
});
// R2 fix (guard blocker 2): valid YAML scalars containing colons
// (ISO timestamps, URLs) MUST parse cleanly via the `yaml` package
// and the artifact MUST be included. The R1 local parser excluded
// these as "malformed" because of the inner colons.
it("includes ready artifacts with ISO timestamp frontmatter (POC YAML parity)", async () => {
writeFileSync(
join(tmp, "with-timestamp.md"),
"---\nstatus: ready\nclaimed_at: 2026-05-03T05:15:25Z\n---\n# Body\n",
);
writeFileSync(
join(tmp, "malformed.md"),
"---\nstatus: ready\nbroken: value: still broken\n---\n",
);
const out = await artifactPoolReadyPolicy.evaluate(
makeJob({
context: { pools: [{ path: tmp, include_statuses: ["ready"] }] },
}),
);
expect(out.action).toBe("send");
if (out.action !== "send") return;
expect(out.message).toContain("with-timestamp.md");
expect(out.message).not.toContain("malformed.md");
expect(out.message).toMatch(/has 1 actionable/);
});
it("includes ready artifacts with URL frontmatter (POC YAML parity)", async () => {
writeFileSync(
join(tmp, "with-url.md"),
"---\nstatus: ready\nsource_url: https://example.org/path\n---\n# Body\n",
);
const out = await artifactPoolReadyPolicy.evaluate(
makeJob({
context: { pools: [{ path: tmp, include_statuses: ["ready"] }] },
}),
);
expect(out.action).toBe("send");
if (out.action !== "send") return;
expect(out.message).toContain("with-url.md");
});
it("includes ready artifacts with quoted-string frontmatter containing colons", async () => {
writeFileSync(
join(tmp, "with-quoted-colon.md"),
"---\nstatus: ready\ntitle: \"Phase C: things\"\n---\n",
);
const out = await artifactPoolReadyPolicy.evaluate(
makeJob({
context: { pools: [{ path: tmp, include_statuses: ["ready"] }] },
}),
);
expect(out.action).toBe("send");
if (out.action !== "send") return;
expect(out.message).toContain("with-quoted-colon.md");
});
it("supports configured ignore_names (POC parity)", async () => {
writeFileSync(join(tmp, "skip-me.md"), "---\nstatus: ready\n---\n");
writeFileSync(join(tmp, "include-me.md"), "---\nstatus: ready\n---\n");
@@ -194,6 +194,107 @@ describe("WatchdogScheduler (PL-004 Phase C)", () => {
expect(sched.isRunning()).toBe(false);
});
// R2 fix (guard blocker 1): the scheduler MUST consult
// scan_interval_seconds when set; interval_seconds is a fallback only.
// These tests use distinct values so a regression that uses
// interval_seconds will fail (unlike R1's equal-valued active-wake test).
it("isDue uses scan_interval_seconds (=30) over interval_seconds (=600) — due after 45s elapsed", () => {
const job = jobsRepo.register({
policy: "periodic-reminder",
specYaml: "policy: periodic-reminder\ntarget:\n session: a@rig\nmessage: x\n",
targetSession: "a@rig",
intervalSeconds: 600,
scanIntervalSeconds: 30,
registeredBySession: "ops@kernel",
});
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
const updated = jobsRepo.getByIdOrThrow(job.jobId);
expect(isDue(updated, Date.now())).toBe(true);
});
it("isDue uses scan_interval_seconds (=600) over interval_seconds (=30) — NOT due after 45s elapsed", () => {
const job = jobsRepo.register({
policy: "periodic-reminder",
specYaml: "policy: periodic-reminder\ntarget:\n session: a@rig\nmessage: x\n",
targetSession: "a@rig",
intervalSeconds: 30,
scanIntervalSeconds: 600,
registeredBySession: "ops@kernel",
});
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
const updated = jobsRepo.getByIdOrThrow(job.jobId);
expect(isDue(updated, Date.now())).toBe(false);
});
it("runTickNow does NOT invoke policy nor write history during scan_interval_seconds not-due window", async () => {
// scan_interval_seconds=600 with interval_seconds=30 (the discriminating
// pair: a scheduler that ignores scan_interval_seconds would be due
// after 30s and would invoke the policy + write history). Test asserts
// policy is NOT invoked through the scheduler boundary AND that no
// sent history row was written.
let policyCalled = 0;
const job = jobsRepo.register({
policy: "periodic-reminder",
specYaml:
"policy: periodic-reminder\ntarget:\n session: counted@rig\nmessage: counted\n",
targetSession: "counted@rig",
intervalSeconds: 30,
scanIntervalSeconds: 600,
registeredBySession: "ops@kernel",
});
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
const engine = new WatchdogPolicyEngine({
jobsRepo,
historyLog: log,
eventBus: bus,
deliver: async (req) => {
policyCalled += 1;
deliveries.push(req);
return { status: "ok" };
},
});
const sched = new WatchdogScheduler({ jobsRepo, policyEngine: engine });
const historyCountBefore = log.countForJob(job.jobId);
await sched.runTickNow();
expect(policyCalled).toBe(0);
expect(deliveries.length).toBe(0);
expect(log.countForJob(job.jobId)).toBe(historyCountBefore);
expect(jobsRepo.getByIdOrThrow(job.jobId).lastEvaluationAt).toBe(fortyFiveSecondsAgo);
});
it("runTickNow DOES invoke policy when scan_interval_seconds elapsed even if interval_seconds large", async () => {
let policyCalled = 0;
const job = jobsRepo.register({
policy: "periodic-reminder",
specYaml:
"policy: periodic-reminder\ntarget:\n session: counted@rig\nmessage: counted\n",
targetSession: "counted@rig",
intervalSeconds: 600,
scanIntervalSeconds: 30,
registeredBySession: "ops@kernel",
});
const fortyFiveSecondsAgo = new Date(Date.now() - 45_000).toISOString();
jobsRepo.recordEvaluation(job.jobId, fortyFiveSecondsAgo, true);
const engine = new WatchdogPolicyEngine({
jobsRepo,
historyLog: log,
eventBus: bus,
deliver: async (req) => {
policyCalled += 1;
deliveries.push(req);
return { status: "ok" };
},
});
const sched = new WatchdogScheduler({ jobsRepo, policyEngine: engine });
await sched.runTickNow();
expect(policyCalled).toBe(1);
expect(deliveries).toEqual([{ targetSession: "counted@rig", message: "counted" }]);
expect(log.countForJob(job.jobId)).toBe(1);
});
it("recovers schedule across restart via SQLite (new repo + new scheduler picks up active jobs)", async () => {
jobsRepo.register({
policy: "periodic-reminder",