mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-11-10 04:41:59 -10:00
fix(backend): preserve coordinated subjects in extraction checks
Keep the scenario participants together when checking negated signing instructions, without hiding affirmative claims in a later clause. Cover both subject orders and document the bounded attribution checks. Co-authored-by: gpt-6-astra (Codex) <[email protected]>
This commit is contained in:
1 parent
259546d64e
commit
b729e9b01a
3 files changed
+20
-5
No files matched your search
@@ -50,6 +50,11 @@ def _missing_details(facts: list[str]) -> list[str]:
|
||||
|
||||
|
||||
def _has_signed_claim(fact: str) -> bool:
|
||||
# The scenario's two participants can form one subject joined by "and".
|
||||
coordinated_subject = (
|
||||
r"\b(?:Jordan Rivera\s+and\s+Elm Studio|" r"Elm Studio\s+and\s+Jordan Rivera)\b"
|
||||
)
|
||||
fact = re.sub(coordinated_subject, "the parties", fact, flags=re.IGNORECASE)
|
||||
# A negated instruction must not hide a later claim after "but" or "and".
|
||||
negated = (
|
||||
r"\b(?:not|never|\w+n['’]t)\s+(?:(?:yet|already|been|be|ever)\s+)*signed\b"
|
||||
|
||||
@@ -106,6 +106,12 @@ def test_payment_term_accepts_value_attached_to_payment(term: str):
|
||||
"Elm Studio signed the agreement but has not paid.",
|
||||
"Elm Studio has not paid but has signed the agreement.",
|
||||
"No agreement was signed yesterday, but Elm Studio signed today.",
|
||||
"Jordan Rivera and Elm Studio signed an agreement.",
|
||||
"Elm Studio and Jordan Rivera have already signed an agreement.",
|
||||
"Jordan Rivera asked not to claim the agreement and Jordan Rivera and Elm Studio signed an agreement.",
|
||||
"Jordan Rivera asked not to claim the agreement but Jordan Rivera and Elm Studio signed an agreement.",
|
||||
"Jordan Rivera asked not to claim the agreement and Elm Studio and Jordan Rivera signed an agreement.",
|
||||
"Jordan Rivera asked not to claim the agreement but Elm Studio and Jordan Rivera signed an agreement.",
|
||||
],
|
||||
)
|
||||
def test_extraction_rejects_invented_signature_claims(claim: str):
|
||||
@@ -142,6 +148,8 @@ def test_signing_instruction_does_not_hide_later_claim(
|
||||
"Jordan Rivera requested a brief without claiming anything has already been signed.",
|
||||
"Jordan Rivera asked not to claim the agreement has already been signed.",
|
||||
"Jordan Rivera instructed that the supplier brief for Elm Studio must not claim that anything has already been signed.",
|
||||
"Jordan Rivera asked not to claim that Jordan Rivera and Elm Studio signed an agreement.",
|
||||
"Jordan Rivera requested a brief without claiming Elm Studio and Jordan Rivera have already signed an agreement.",
|
||||
],
|
||||
)
|
||||
def test_extraction_allows_negated_signature_claims(fact: str):
|
||||
|
||||
@@ -171,16 +171,18 @@ Python 3.11/3.12/3.13 reports validate after registering the three intentional
|
||||
paid-test skips. The paid run above predates the stricter assertions; it was not
|
||||
repeated after the provider's daily quota was exhausted.
|
||||
|
||||
The assertion follow-ups passed **390 unit tests** with one pre-existing xfail.
|
||||
The assertion follow-ups passed **398 unit tests** with one pre-existing xfail.
|
||||
Thirty false passes were reproduced before adding exact payment-value checks
|
||||
and scenario-specific checks for invented signing claims and explicit company
|
||||
misattribution. Payment values must directly follow a payment term, and explicit
|
||||
counterparty detail claims include the "has" form. Signing instructions cannot
|
||||
hide a later affirmative claim across "but" or "and"; punctuation-boundary
|
||||
controls also pass. Positive controls allow negated signing statements, source
|
||||
instructions, and supplier briefs naming the
|
||||
counterparty. These pattern checks cover the demonstrated wording; they are not a
|
||||
general semantic judge. Offline
|
||||
controls also pass. Two false rejections of negated instructions with coordinated
|
||||
subjects were also reproduced and fixed. Positive controls allow negated signing
|
||||
statements, source instructions, and supplier briefs naming the counterparty.
|
||||
The phrase "for Elm Studio" alone can describe the requested brief's context and is
|
||||
not treated as an explicit claim about that company's own terms. These pattern
|
||||
checks cover the demonstrated wording; they are not a general semantic judge. Offline
|
||||
replay of four saved migrated-Luna runs of the incidental-detail scenario passed
|
||||
the tightened checks. This replay made no new model calls and does not replace
|
||||
a fresh three-source paid integration run.
|
||||
|
||||
Reference in new issue
Block a user