fix(backend): preserve coordinated subjects in extraction checks

Keep the scenario participants together when checking negated signing
instructions, without hiding affirmative claims in a later clause. Cover
both subject orders and document the bounded attribution checks.

Co-authored-by: gpt-6-astra (Codex) <[email protected]>
This commit is contained in:
Nicholas Tindleandgpt-6-astra committed 2026-10-19 08:29:49 -05:00
1 parent 259546d64e
commit b729e9b01a
3 files changed
+20 -5

No files matched your search

@@ -50,6 +50,11 @@ def _missing_details(facts: list[str]) -> list[str]:
def _has_signed_claim(fact: str) -> bool:
# The scenario's two participants can form one subject joined by "and".
coordinated_subject = (
r"\b(?:Jordan Rivera\s+and\s+Elm Studio|" r"Elm Studio\s+and\s+Jordan Rivera)\b"
)
fact = re.sub(coordinated_subject, "the parties", fact, flags=re.IGNORECASE)
# A negated instruction must not hide a later claim after "but" or "and".
negated = (
r"\b(?:not|never|\w+n['’]t)\s+(?:(?:yet|already|been|be|ever)\s+)*signed\b"
@@ -106,6 +106,12 @@ def test_payment_term_accepts_value_attached_to_payment(term: str):
"Elm Studio signed the agreement but has not paid.",
"Elm Studio has not paid but has signed the agreement.",
"No agreement was signed yesterday, but Elm Studio signed today.",
"Jordan Rivera and Elm Studio signed an agreement.",
"Elm Studio and Jordan Rivera have already signed an agreement.",
"Jordan Rivera asked not to claim the agreement and Jordan Rivera and Elm Studio signed an agreement.",
"Jordan Rivera asked not to claim the agreement but Jordan Rivera and Elm Studio signed an agreement.",
"Jordan Rivera asked not to claim the agreement and Elm Studio and Jordan Rivera signed an agreement.",
"Jordan Rivera asked not to claim the agreement but Elm Studio and Jordan Rivera signed an agreement.",
],
)
def test_extraction_rejects_invented_signature_claims(claim: str):
@@ -142,6 +148,8 @@ def test_signing_instruction_does_not_hide_later_claim(
"Jordan Rivera requested a brief without claiming anything has already been signed.",
"Jordan Rivera asked not to claim the agreement has already been signed.",
"Jordan Rivera instructed that the supplier brief for Elm Studio must not claim that anything has already been signed.",
"Jordan Rivera asked not to claim that Jordan Rivera and Elm Studio signed an agreement.",
"Jordan Rivera requested a brief without claiming Elm Studio and Jordan Rivera have already signed an agreement.",
],
)
def test_extraction_allows_negated_signature_claims(fact: str):
@@ -171,16 +171,18 @@ Python 3.11/3.12/3.13 reports validate after registering the three intentional
paid-test skips. The paid run above predates the stricter assertions; it was not
repeated after the provider's daily quota was exhausted.
The assertion follow-ups passed **390 unit tests** with one pre-existing xfail.
The assertion follow-ups passed **398 unit tests** with one pre-existing xfail.
Thirty false passes were reproduced before adding exact payment-value checks
and scenario-specific checks for invented signing claims and explicit company
misattribution. Payment values must directly follow a payment term, and explicit
counterparty detail claims include the "has" form. Signing instructions cannot
hide a later affirmative claim across "but" or "and"; punctuation-boundary
controls also pass. Positive controls allow negated signing statements, source
instructions, and supplier briefs naming the
counterparty. These pattern checks cover the demonstrated wording; they are not a
general semantic judge. Offline
controls also pass. Two false rejections of negated instructions with coordinated
subjects were also reproduced and fixed. Positive controls allow negated signing
statements, source instructions, and supplier briefs naming the counterparty.
The phrase "for Elm Studio" alone can describe the requested brief's context and is
not treated as an explicit claim about that company's own terms. These pattern
checks cover the demonstrated wording; they are not a general semantic judge. Offline
replay of four saved migrated-Luna runs of the incidental-detail scenario passed
the tightened checks. This replay made no new model calls and does not replace
a fresh three-source paid integration run.